I've only played WoW for a little time and at low level, so I'm mostly speculating.
I think the difference is precisely because their wasn't any dungeon finder.
Like it or not, the Duty Finder implies more flexibility when you design dungeons. You can't make a dungeon where you'll assume that every party will have the "best" tank, the "best" healer or the "best DPS". So, their mechanics are focused on something else, wether it be dodging, focusing on the right target, etc...even the DPS check that you find in Duty Finder content are pretty forgiving, because they need to be doable by any party setup.
This is also why Coil was nerfed.
Why did they make ADS Paralyze weaker ? Because it's not uncommon to end with a DF party without any BRD of PLD so that you can't silence it.
Why didn't Renaud killed you in one hit ? Because you can end with a party without any ranged DPS, so you need to pick up Renauds at close range.
Why is Alex far easier than Coil at launch ? Because it's in the Duty Finder.
The ONLY content where optimal party setup is a real thing is out of the Duty Finder, so for now, Rav and Bis EX, and Alexander Savage.
Lots of people probaly didn't clear Rav/Bis Ex before gathering several i180 pieces, or even 190 (I cleared all floors of Alex before Bis EX), and now, the DPS check is far less demanding, as long as you know how to play your job.
I'd take a really competent BRD or MNK over a bad BLM or DRG, even if the "numbers" say that their max DPS is slightly weaker.
So, again, except for really hard and demanding content, a small discrepancy between jobs would not be a big thing, since it's ALREADY not a big thing.
And for those who participate in this demading content, please theorycraf all you want so you'll find what specs is the absolute best for each job. I'm sure you'll even like searching for it...
That's why I'm saying the fear of imbalance is a fallacy. Because it's already in the game (And it will always be, unless you have only one job per role) and yet the games work fine. And adding more jobs will create more discrepancy too, so your fear should be the same.