You simply aren't going to get people repeating the same content without some measure of rewards for doing so, regardless of how well it's designed. They'll inevitably grow bored. In a single player game, which you keep alluding towards, this works because they have no financial motivation beyond your initial purchase. MMOs are trying to retain a monthly sub. They don't want you leaving, but can't develop content fast enough to keep up with how quickly people devour it. Hence why grind styled accomplishments exist.
There will still remain a "best" way to approach a given situation. If you add elemental weaknesses and make Black Mage the most consistent in dealing damage against said weakness, it immediately becomes the de facto caster teams want. If you make a boss weak to Piercing, DRG/BRD/MCH are now slotted as the best. Look at how many people bring 3-4 tanks into Rathalos. Why? They can shrug off his dives far better than DPS jobs can and laugh at the stack mechanic. In that sense, they're the best option, which vaults them into the "default" positions.
Yes and no. FATEs and such do need a design overhaul, but they're typically ignored because:
1) The devs overused them to the point people became fed up.
2) They offer very little rewards, thus no incentive to bother.
And how do you accomplish that? How do you make Ridorana worth exploring outside of your first time? Twenty three people aren't going to wait around while you look at the map and check every little room. In fact, look at what happened in Rabanastre with the little notes underwater. People started getting annoyed or outright pulled the bosses because they didn't care to wait. If you force them to, well, you get the mess that has been Praetorium and Castrum, where the devs practically had to throw massive tome rewards just to get people into the MSQ roulette. While you may look at Ridorana and want to explore every little nook and cranny, a good number of people aren't going to care. They want to kill the bosses and collect their loot.
A single player game, among my favourites of all time, actually. Nevertheless, it could do things a multiplayer game cannot because it isn't dependent on anyone else except the one person it's trying to entertain. You can take your time exploring, messing around with strange builds that are nowhere near good or any number of things because it's only your time being used. I used to spend hours upon hours playing Blitzball simply because I liked it. (No, Yoshida, that isn't an endorsement.) If you attempt an unorthodox build in a multiplayer game, well, you're likely playing something inferior which hinders group progression. Now if everyone else is cool with that, have at it. I, personally, wouldn't be too pleased because your fun is wasting my time. There's a reason people get uppity about healer DPS and tank stance, after all.
And nothing you suggested will change that. Sure, people may be excited to spec DRG a certain way at first, but the novelty will inevitably wane once superior builds are discovered. People are bored because the content hasn't been up to par and very little has changed. Presently, we have them throwing huge amounts of money towards an old school design no one really cares for nowadays. Meanwhile, people have been clamoring for Mythic+ styled content for years. WoW hasn't changed its core systems in over a decade, but has tried different ideas in addition to them. FFXIV doesn't. The main issue with the latter is they play everything safe; relying entirely on a formula. That model works... if combined with different ideas that feel fresh ala WoW. You don't have to overhaul everything.