I used to be the same way thinking I can't do savage which is why I skipped out on Gordias and A little of Midas. Though now that everything is basically creator level I think I can not be stressed so much and have fun. I really do enjoy raiding but i'm not an excessive raider like 3 times a week. I much prefer twice a week and I'm not even sure how people raid everyday without pulling their hair out. Either way, I feel like savage isn't challenging but just the right fit for me atm to have something to work towards. Yeah, gear gets pointless but if that's the case why grind tomes at all?
Bottom line is.... get people you know or are friends with and play with them if you are so anxious. At least that way nobody will be pointing fingers and such.
