OP, you bring up good and interesting points, many of which I've pondered myself as well. I tend to agree with much of what has already been said by others. Some of my favourite RPGs of all time threw balance out the window and ended up being better for it - Final Fantasy Tactics, to name but one.
However, and I'm not entirely sure if this has been brought up or not, some of the comparisions for more streamlined and superficial experiences that have come up concern single player games, e.g. Final Fantasy XII and XIII. And therein lies a part of the problem, I think. While I'm totally in favour of a less balanced but more varied, in-depth and engaging experience for single player games (I loved breaking the hell out of FFT, lol), this is an MMO we're dealing with here. This means that the variety of the player base has to be considered. You have more and less experienced players, some players play the game really well, others not so much, and so on.
If you design a single-player game, I tend to think it's easier to strike a balance between, well... balance and variety because what the player does with what they've been given is entirely up to them. No other person is involved in the experience. This does not apply to multi player games, where not only a player's stance on the game and its systems, but also on other people playing the game have to be taken into account.
So, for example, if you have one class that is, say, really difficult to play but a blast and really powerful when done right, you'll usually end up with two (or more) camps: Camp number one who plays the class and does so well will have tons of fun playing it. Another set of people, who plays the class as well but perceives it as too difficult to be able to realize its full potential will say that it's too hard to play and should be simplified for maximum output. Yet another group, who prefers a more straightforward, but no so powerful class will say that class X is OP and should be nerfed because 'Waaah, my class doesn't have as much output as class X does'.
You can extend this argument to other areas, for example class combinations in high-end content and the fact that people have been openly harassed in PF/RF for picking the 'wrong' class. Really, you can't make that stuff up.
What I mean to say is this: The debate of balance vs. variety in games is a hard enough one already, as the two often come at each other's expense, but with so many classes and people who play them, and subsequently many, many interests at odds with each other, I just think it's the easier and less stressful path for the devs to balance, and thus homogenize, as much as they can, both from a game design and business standpoint. Things like depth, variety and class identiy will necessarily suffer from this, and this is indeed a problem, but I feel that the more discerning player base who favours more depth and variety while accepting a certain loss of balance will always be in the minority.