Job Balancing 6.08 (still needs work)

[Gruntler]

Then our balance is already perfectly sufficient. Every job can clear with little difficulty essential to its kit. Done.

The ability of high-tier players to clear with everything does not imply it's easier for lesser skilled players to have equal proficiency in clearing with every job. Job balance matters more than at the absolute top end.

utility is only as valuable as the average improvement to clear times (including whether you clear at all) it produces. If it does not increase your chances of clearing and/or reduce clear times, it has done nothing.

Which is not something FFLogs has any capacity to measure, but in-game tools can actually provide input for. However, DPS also only matters for balance with how it affects your ability to clear or reduce clear times. Which gets to the more important point: The vector of balance is capacity to clear across different play levels.

[quote[The criteria in the post you're quoting already follows from consideration of spread, deviation, and outliers. But let's be clear here: an entire 10% of players is not going to be an outlier. As you go lower, moreover, the parses are that much less representative of their kits, as opposed to simply varied, and often short-lived, forms of mistake.[/quote]

But people don't consider spread, deviation or outliers. Case in point:

But let's be clear here: an entire 10% of players is not going to be an outlier.

An outlier is based upon deviation, and it is quite possible for 10% of players to be deviated too much to be counted as normalized data.

How does the phrase "to be clear" have anything to do with clears? That's literally the only the time that word appears in my post -- within that phrase. Where is this strawman coming from?

You're appealing to FFLogs data. FFLogs data ONLY includes clears. That introduces sample bias that excludes aspects of balance (including difficulty to attain clear) that are more valuable and relevant to job balance than speedruns. What you said or what phrases you used is irrelevant--you're appealing to FFLogs, and this is a criticism of that appeal with regards to data-validity.

[Shurrikhan]

You're appealing to FFLogs data. FFLogs data ONLY includes clears. That introduces sample bias that excludes aspects of balance (including difficulty to attain clear) that are more valuable and relevant to job balance than speedruns. What you said or what phrases you used is irrelevant--you're appealing to FFLogs, and this is a criticism of that appeal with regards to data-validity.

Which would be relevant only if clearing were significantly harder on some jobs than other. But it's not.

An outlier is based upon deviation, and it is quite possible for 10% of players to be deviated too much to be counted as normalized data.

If it's deviation sufficiently differed from those immediately above or, more importantly, below. But it doesn't.

We don't see any such leap until after the 95th percentile, which is not what I'm suggesting we balance around.

All that looking at 90th percentile gives you is a reasonable view of a kit's strength among a reasonable level of error; the impact of those errors have more or less standardized, their deviations shrunk. The jobs are still necessarily played perfectly, but at least those values indicate what a kit, for the most part, is capable of, not just an average player still learning how to use it.

And you absolutely should be balancing jobs primarily around their kits, not just how easy they are to push from poor performance to halfway decent.

Which is not something FFLogs has any capacity to measure

I never said it did. That's exactly why I said it'd require us to comb the context to take those things into account. Our evaluation must, further, include those things. To see whether they have practical chance of value, you look at better runs first -- those who'd actually know how to use them. Then you overpower them slightly in accordance with how difficult that utility's value is to exploit/coordinate.

Let's put it another way. X job is taxed Y value for having Z utility. But in practice, that utility doesn't save even a single GCD of healing except in the very most optimized runs. That will not be sufficient. That it can save a GCD in an optimized run should be noted so that we at least have some idea to the form of its potential value, but its tax should be diminished to reflect those constraints.

[Gruntler]

If it's deviation sufficiently differed from those immediately above or, more importantly, below. But it doesn't.

This is not a strong point. This would be expected in a normal distribution.

We don't see any such leap until after the 95th percentile, which is not what I'm suggesting we balance around.

You don't have the standard deviation to claim this.

All that looking at 90th percentile gives you is a reasonable view of a kit's strength among a reasonable level of error;

Or put differently 'By imposing a sample bias against the skill floor of a job, you exclude the skill differential as a part of balance consideration.' Which is not good thinking in terms of balance, and is frankly, terrible data-analysis praxis.

And you absolutely should be balancing jobs primarily around their kits, not just how easy they are to push from poor performance to halfway decent.

The answer to, in any data analysis, for any reason, 'which percentile should I focus on' is never 'only one percentile' unless the question is, and only is, 'what is that percentile's value?'

Then you overpower them slightly in accordance with how difficult that utility's value is to exploit/coordinate.

True, if, and only if, you're balancing towards post-mastery of content, and not the journey of mastery of that content. There's this idea that PvE balance somehow follows the same rules of PvP balance, where you're trying to ensure a fair competition in some sort of adversarial tournament. This game doesn't really work that way. The game balance itself is more concerned with how players interact with and succeed or fail against the content.

If I want to balance-check difficult mechanic, why would I even care about the throughput of the x players who have defeated it and have it down to muscle memory? That's irrelevant to my needs!

I'd look at the Y who haven't yet defeated it, and I'd look to see if the comparison between X and Y is a number that is considered reasonable. That will tell me if a job is struggling or not, in a more direct way than FFLogs.

except in the very most optimized runs.

But that's the problem. "Optimized run." is an 'invalid metric.' It does not paint a strong picture of the information you seek (job balance vs pve content) You would choose that other information.

You have people in the community who don't understand the concept of 'data validity' and so they come up with very long, elaborate justification why they can use the x percentile. This is easy. It's also lazy.

Game balance isn't about raw potential, it's about difficulty, and raw potential's not a measure of access to potential in pre-mastery learning, which is where the game balance lives. You need a wider spread for that information.

[Kazimere]

snip

Potential of a job is all that matters. They can't balance the game around people who can't be bothered to learn their job. Outside of a disability (cognitive or physical), there isn't one job in this game that is hard to play at a competent level. Are you going to be parsing 100s with them? No, but then again most people don't bother to plug their rotation into spreadsheets (I don't blame them). Fact of the matter is that every single job in the game can execute a high purple without needing to do that, and that is more than enough to come in well under enrage. I'm pretty sure they balance encounters around the 50-75 anyway, which is pretty easy to achieve and doesn't require mastery of any job.

[Shurrikhan]

Or put differently 'By imposing a sample bias against the skill floor of a job, you exclude the skill differential as a part of balance consideration.'

As you should so long as you are balancing a kit, rather than attempting to force equity from varying forms of mistake.

The answer to, in any data analysis, for any reason, 'which percentile should I focus on' is never 'only one percentile' unless the question is, and only is, 'what is that percentile's value?'

I literally said as much. I have never advocated for caring only about a single percentile. The post at the start of this conversation asks that we consider what can be done to tighten gaps elsewhere. The difference is merely that I'm not willing to condemn diversity in job complexity or learning curves, which act to increase the zone of attraction available to players so that there's a higher chance of someone having something they enjoy at their present level (which is not limited to aesthetic, performance, or even playflow, alone) to achieve the same tightness for every span of player. It's (A) impossible, and (B) horribly constraining to job design.

I've used the 90th percentile only to point out at what degree of skill relative to others using one's job within the first couple weeks of progression where, given what I consider a good balance in design philosophy between job diversity and equity, I'd expect job performance to be tightest.

But that's the problem. "Optimized run." is an 'invalid metric.'

Optimized use is what determines what X is at best capable of. I have repeatedly now said that such is not the be all and end all of tuning.

You would choose that other information.

When, per the premise you're following her, the value of the utility in any other run is nil, then that leaves you with only two options: (A) to make that utility perfectly free, overtuning it for any party who knows how to put it to use, or (B) remove the utility, favoring equity over identity, available depth, etc.

I'd look at the Y who haven't yet defeated it, and I'd look to see if the comparison between X and Y is a number that is considered reasonable. That will tell me if a job is struggling or not, in a more direct way than FFLogs.

Why assume that the devs couldn't possibly have access to those numbers? I've already noted

You have people in the community who don't understand the concept of 'data validity' and so they come up with very long, elaborate justification why they can use the x percentile. This is easy. It's also lazy.

Then perhaps... give something yourself, beyond 'everything should be balanced everywhere from everyone' as if such had no destructive implications on job design philosophy?

Game balance isn't about raw potential, it's about difficulty, and raw potential's not a measure of access to potential in pre-mastery learning, which is where the game balance lives.

Which is exactly why I mentioned changes across percentiles...

I don't know why you're acting like I want to balance things only for perfectly theorycrafted, perfectly executed robotic performances. I don't. But until you have at least some rough line in the sand, you're as likely to overbuff all but the easiest few options, leaving jobs like Machinist crippled once its competitors move more than a few baby steps towards mastery. I am merely being mindful of both this and the impact that having diverse kits and thereby learning kits, something I do not want to give up, will have upon job equity at different points in players' learning of those jobs.