That's a matter of opinion... and one I do not share. Players can progress character power without raiding, and the gap in gear power between those who raid and those who don't is very small compared to other mmo's that accentuate raiding as their primary endgame.
If the main thing someone cares about is raiding, it's not surprising that they might not acknowledge the other "endgame" content that FFXIV has to offer. Just because you consider raiding "the only real endgame" FFXIV has, doesn't mean that's actually the case. I personally don't feel it is and I am fairly confident that I am not alone in that.
Raiding (by which I mean savage) is niche. The amount of players who engage in it is minuscule compared to those who don't. So if you're saying that raiding is the "only real endgame" you're also saying that the majority of those playing FFXIV don't engage in the "only real endgame". So what, pray tell, are they all doing then?