I think a huge part of the problem is that there isn't really anything to gain from doing the savage raids, unless you're extremely into min/maxing stuff. The average player looks at the rewards from the raids, then looks at the stuff they can get from other sources, looks at what there is to do outside of raids that need gear, shrugs and says "why bother with the hassle of doing the savage stuff, for rewards that will never be useful for anything?" and just doesn't bother with even trying to get into the raiding comunity.

I know I reason like that, I know that I'll be able to get gear of equal value (or very close to) without doing the savage modes if I just wait for a while, and there's nothing else in the game that I'd ever need that gear to complete.

This means that a large part of the raiding cuminity in this game is made up of the min/max kind of people, and they tend to be pretty picky with who they raid with.