Well that is one problem I have with gameplay built around rewards in general, though putting that aside, it's not always true that people aim to do things as efficiently as possible. Have you never tried to cook something yourself just to try it? Have you ever been out and taken a turn off the path you were intending to go down just to explore? When players are being gatekept by things like XP, drip fed gear, and timed lockouts I'm not surprised that they try to minimize the time it takes to reach the end. The experience isn't the reward, it's a time sink to prevent you from moving on even if you're starting to get bored.
This is usually the opposite of what a MMO game uses to entice new players. The pitch is typically trying to sell prospective players on the idea of going on a grand adventure with surprises and interesting events along the way. FF14 definitely does this in its advertising and while the MSQ has its fair share of interesting plot, it along with the rest of the game is a bit too rigid to deliver on the promise that the advertising tries to sell. You don't get a world to explore. You get a predetermined story to sit through. That story can be good, it can even be fantastic, but it will never be the same kind of adventure that is being used to attract players in the first place. At least that's how I see it.
What's needed is uncertainty. As great as procedural layouts would be, they aren't the only design choice that fosters variety. Fixed dungeons like those we have now could be improved with enemies and items in different positions. You mentioned earlier that players would try to optimize spawn points, but that's only going to work if those spawn points are very limited and predictable. Player starting points would have to be limited for most existing dungeon layouts, but enemy placement would be nearly entirely free. Interactions with the environment is another idea, what if a path that was initially clear is blocked off after a boss fight because of fallen debris?The only way to have a true explorative experience every time you go into a dungeon is to have a new layout for the dungeon each time. There has been mmos that have played around with procedurally generated dungeons. Anarchy Online, City of Heroes/Villains, even deep dungeons in FFXIV. All have been, in general, fairly lacking, mostly due to modular systems that MMOs use to do the generating.
Dungeon variety is only part of the problem though because if everything is a reward treadmill the content eventually becomes tedious. I think what MMO's need is to lean more heavily into of the promise of a world to adventure in. Tell a story that captures players' attentions and gets them invested and then build content that gives those players some agency over the world. Have players grown attached to the losing side in a war and want to help them? Instead of locking the fate of the conflict to the narrative, let player actions have some impact. They can fight the aggressive side or craft items for the losing side and eventually if enough is done the aggressor can be persuaded to give up.
This is absolutely more complex than using copy and paste on similar instances with minor aesthetic changes over and over while artificially limiting progress behind many different progress or resource bars. I'm well aware of that. I still want the more fun option.