The "same" fight doesn't have to function identically for 4 people as it would 8, though. Many fights already scale their mechanics based on the number of survivors. (Heck, there's quickly a complaint thread up any time a trial fails to do that, because it makes it harder to underman farm in the next expansion when all challenge is gone from it either way.)
You don't need necessarily to design a fight for one set composition. You can generate the conditions for a skill table's formation, rather than using just one actual skill table for all comers.
Now, IF you were to design for just a 1/2/1 or 2/4/2 experience, that would then have the problems you've mentioned -- a light party design would be forced to have more concentrated intervals of damage or more lenient rest damage between, because all ability mitigation is dealt over half the uptime as in a full party -- forcing use of Cover skills or for those tankbusters to be dealt as multiple hits as to be swapped between mid-chain (your easiest standard fix for light party upscaling). But there's simply no need to fetter oneself in that way. You can have x adds with y damage at z frequency (timings, usually associable with z skillsets' coverage), whatever you need.
My intended emphasis here was actually on toolkit synergies between players, which are comparatively lacking in XIV due to near-complete absence of impactful short-term throughput goals in outside of eHP (party or target survival) and specified DPS checks. The closest I can think of in XIV to the above are things like barriers (used to nullify the threat of knockbacks or debuff infliction that would otherwise limit the affected's tactical options), Cover, Rescue, someone taxi'ing Aetherial Manipulation, etc. From a fight design perspective itself... (see above).
Having said all that... I do feel like so long as you don't intend for Eureka to feel like an open exploration zone for a larger data center or server community experience, but rather just an isolated, dedicated team without any added hassles, designing primarily towards 8-mans is ideal. I'd just prefer not to be forced to take a 2/4/2 composition, and I'd prefer that my way out of that be through more than just stat leniency.
I wholly agree that FATEs could actually end up tremendously interesting, if XIV were to have the guts and trust to make them so. To be honest, being an fan of dynamic open-world content I wish we'd see that kind of stuff get more development time. And yet, I'd probably be pissed if I were to hear that even something as mediocre as one our expert roulette dungeons were sacrificed in order to create a few new FATEs, if only because of how unlikely I find it that they will ever be more than a "set objective; add objective center, if any; set domain(s); set NPC node(s), if any; specify mob models, levels, and names; set progress counter for spawn triggers, if any; set spawn positions or seed pattern" affair where localization for cringe jokes probably takes far more time than the actual design.
But I'll agree absolutely that open world dynamic events, in their full potential, could end up incredibly engaging if well designed.