"Balance team tests the fights themseslves"

[SeiyaSoiya]

I am sure they do extensive testing and don't have any issues clearing the fights
however it really rubs me the wrong way that they pretend as if the dev/tester prog experience is the same as the player prog experience
you cannot and will never convince me that dsr was cleared during testing without the use of dev tools to checkpoint each phase and prog each independently

meanwhile actual players must always go back to phase 2 each wipe.
yes, you didn't need 3rd party tools to clear dsr, you had 1st party tools that are a million times more cheaty and powerful than any 3rd party tool

[Avenger]

I am sure they do extensive testing and don't have any issues clearing the fights
however it really rubs me the wrong way that they pretend as if the dev/tester prog experience is the same as the player prog experience
you cannot and will never convince me that dsr was cleared during testing without the use of dev tools to checkpoint each phase and prog each independently

meanwhile actual players must always go back to phase 2 each wipe.
yes, you didn't need 3rd party tools to clear dsr, you had 1st party tools that are a million times more cheaty and powerful than any 3rd party tool

In one of the producer letters they explained how during testing they use an HP refill mode (refill rather than KO) to avoid having to wipe and restart. This intentionally enables the testing teams to go in blind but learn the fight much faster than regular players would. After they know the fight, the now-expert team confirms that it can be cleared with all the comps in regular player mode at min ilvl and without HP refill. I imagine Yoshi-p (busy as he is) would also use HP refill mode to learn the fights, and would probably clear them with experienced teams (though maybe still at min ilvl for a bit of fun.)

Obviously if they gave this sort of wipe invulnerability to players during progression it would make learning the fights much faster and easier. Players would also probably complain that the fights were too quick and easy to clear.

I think Yoshi-p also explained that the dev/testing teams are good players but not "world class" players, and they're always impressed at how well players optimize the fights (I imagine, without any evidence of course, that they're also grudgingly impressed with players finding and exploiting bugs and unintended behavior, even if it results in major headaches.)

[Packetdancer]

...that it can be cleared with all the comps in regular player mode...

I feel safe in suggesting there is next to zero chance they actually try every possible comp. Just for the first seven, assuming no duplicate jobs, one pure healer/one barrier healer, and one of each role, you have (assuming my top-of-my-head math isn't wrong):

6 possible combinations of tanks (assuming no duplicates)
4 possible combinations of healers (assuming 1 pure healer and 1 barrier healer)
45 possible combinations of 3 DPS (1 out of 5 possible melee, 1 out of 3 possible casters, 1 out of 3 possible physical ranged)

For I think 1080 total possible party comps just for 7 of the 8 slots.

Whichever three DPS are picked, there are 8 possible DPS jobs remaining (4 out of the 5 melee, 2 out of the three caster, or two out of the three physical ranged). Now, not all of those possibilities apply, of course; MNK for your melee role and SAM for the extra is no different than SAM for the melee role and MNK for the extra. But even allowing for that, you have literally thousands of possible party compositions.

The chances that the team hand-plays through the entire tier at minimum ilevel with every single possible standard comp are breathtakingly unlikely, just given the sheer number of possible standard comps. I absolutely believe they did it back in ARR and, say, maybe Heavensward. But now? The chances are very, very slim that they actually have the time and manpower to play through the fight with every single party comp. Especially if there were any balance work to be done afterwards, when you'd then need to play it through on everything again. You almost certainly pick several 'representative' party compositions you assume give decent coverage, and judge the fight balance from those.

Which I suspect is precisely why this problem arose; those representative party compositions were probably insufficiently representative.

[TaleraRistain]

6 possible combinations of tanks (assuming no duplicates)
4 possible combinations of healers (assuming 1 pure healer and 1 barrier healer)
45 possible combinations of 3 DPS (1 out of 5 possible melee, 1 out of 3 possible casters, 1 out of 3 possible physical ranged)

Dating myself here

https://www.youtube.com/watch?v=w0i_ZFlGTVY

Use combinatorics, a faster way to count
Use combinatorics, you can multiply the amount
Use combinatorics, it's so easy to do
Use combinatorics when you need it
Let the numbers work for you

[SeiyaSoiya]

In one of the producer letters they explained how during testing they use an HP refill mode (refill rather than KO) to avoid having to wipe and restart. This intentionally enables the testing teams to go in blind but learn the fight much faster than regular players would. After they know the fight, the now-expert team confirms that it can be cleared with all the comps in regular player mode at min ilvl and without HP refill. I imagine Yoshi-p (busy as he is) would also use HP refill mode to learn the fights, and would probably clear them with experienced teams (though maybe still at min ilvl for a bit of fun.)

Obviously if they gave this sort of wipe invulnerability to players during progression it would make learning the fights much faster and easier. Players would also probably complain that the fights were too quick and easy to clear.

I think Yoshi-p also explained that the dev/testing teams are good players but not "world class" players, and they're always impressed at how well players optimize the fights (I imagine, without any evidence of course, that they're also grudgingly impressed with players finding and exploiting bugs and unintended behavior, even if it results in major headaches.)

that's interesting, is there any link to this liveletter?

[DixieBellOCE]

I am sure they do extensive testing and don't have any issues clearing the fights
however it really rubs me the wrong way that they pretend as if the dev/tester prog experience is the same as the player prog experience
you cannot and will never convince me that dsr was cleared during testing without the use of dev tools to checkpoint each phase and prog each independently

meanwhile actual players must always go back to phase 2 each wipe.
yes, you didn't need 3rd party tools to clear dsr, you had 1st party tools that are a million times more cheaty and powerful than any 3rd party tool

They also play on 0-1 ping on a localized server where clipping GCD's does not exist, therefore having slightly higher damage output than the regular player.

It's the same issue that arised in FFXI when Absolute Virtue was unkillable unless you used an illegitimate method that the dev team would abuse you for doing (And then patch it out of the game).
They tried to prove the playerbase wrong by showing the correct way of dealing with mechanics, while on 0-1 ping. THis caused an outrage and people clearly pointed out that in an ideal 1 ping setting, it was viable.