Quote Originally Posted by Shurrikhan View Post
If it's deviation sufficiently differed from those immediately above or, more importantly, below. But it doesn't.
This is not a strong point. This would be expected in a normal distribution.

We don't see any such leap until after the 95th percentile, which is not what I'm suggesting we balance around.
You don't have the standard deviation to claim this.

All that looking at 90th percentile gives you is a reasonable view of a kit's strength among a reasonable level of error;
Or put differently 'By imposing a sample bias against the skill floor of a job, you exclude the skill differential as a part of balance consideration.' Which is not good thinking in terms of balance, and is frankly, terrible data-analysis praxis.

And you absolutely should be balancing jobs primarily around their kits, not just how easy they are to push from poor performance to halfway decent.
The answer to, in any data analysis, for any reason, 'which percentile should I focus on' is never 'only one percentile' unless the question is, and only is, 'what is that percentile's value?'

Then you overpower them slightly in accordance with how difficult that utility's value is to exploit/coordinate.
True, if, and only if, you're balancing towards post-mastery of content, and not the journey of mastery of that content. There's this idea that PvE balance somehow follows the same rules of PvP balance, where you're trying to ensure a fair competition in some sort of adversarial tournament. This game doesn't really work that way. The game balance itself is more concerned with how players interact with and succeed or fail against the content.

If I want to balance-check difficult mechanic, why would I even care about the throughput of the x players who have defeated it and have it down to muscle memory? That's irrelevant to my needs!

I'd look at the Y who haven't yet defeated it, and I'd look to see if the comparison between X and Y is a number that is considered reasonable. That will tell me if a job is struggling or not, in a more direct way than FFLogs.

except in the very most optimized runs.
But that's the problem. "Optimized run." is an 'invalid metric.' It does not paint a strong picture of the information you seek (job balance vs pve content) You would choose that other information.

You have people in the community who don't understand the concept of 'data validity' and so they come up with very long, elaborate justification why they can use the x percentile. This is easy. It's also lazy.

Game balance isn't about raw potential, it's about difficulty, and raw potential's not a measure of access to potential in pre-mastery learning, which is where the game balance lives. You need a wider spread for that information.