How to mesure SE healer design success?

[GrimGale]

Forums tend to attract a lot of negativity. Most people come here to complain rather than to applaud design choices.

If we were to mesure Shadowbringer's success in terms of healer design, how would you do it?

Would you figure out how many people consider the healer Jobs their "main" jobs?

Would you try to mesure how many people have leveled their healer jobs?

Would you use Parsers as a metric with how many uploads for a certain role?

How would you do it, to have an accurate representation of player approval of the Shadowbringers healer design?

[Aarchie]

Actually, none of them is a good parameter. I am a healer main, who does savage, and have all healers max level. But I still don't think the actual situation is a good healer design at all...
Unless is an actual poll or study, would be really difficult to tell by estimative and the raw data we have available.
But the amount of complaints in the forum and other media is already indicative (even not being a perfect reflection of all the player base)

[Semirhage]

That's a complicated metric to gather. I wouldn't say it's a single number that points to success or failure. Each individual metric has its own shortcomings. A survey would help so long as the questions weren't worded to elicit specific results. Even with a survey, there's going to be a notable difference between "I've played this job for 10 hours and don't see any issues with it" and "I've played this job for 500 hours and its flaws are obvious to me". This would be true for any role.

Would you figure out how many people consider the healer Jobs their "main" jobs?

This is probably the best of the suggested metrics. Its downside is that it doesn't account for people uncomfortable with swapping out of the role even if they hate it.

Would you try to mesure how many people have leveled their healer jobs?

This is the Eureka trap. They gated multiple incentives behind leveling a job of each role. Getting something to 80 and then never touching it again doesn't indicate something is designed well.

Would you use Parsers as a metric with how many uploads for a certain role?

This is the Eureka trap to a lesser degree. Savage requires you enter with a certain party composition, or you just can't clear. You're going to need to drag two healers through, even if they're not thrilled with their tools.

I think one of the worst obstacles Square has to understanding whether they've designed healers well is that they clearly don't have anyone on the job design team who mains one. If you're not running daily roulettes with a job, taking it through Extreme trials and Savage, playing around in Ultimate, running the MSQ...if you don't play something enough that you understand why someone might find your job boring even if you disagree with it, you're missing out on some important context in the end.

[WaxSw]

Honestly satisfaction polls and maybe counting the amount of posts complaining vs other expansions (relative to the amount of players that play said role) would be a valuable metric and both can misleading too. I feel that the best way to gauge the success would be mixing several sources of data (satisfaction polls+ social media + forum post, for example) and get a conclusion from there, either way you dont need to be an expert analyst to realize that the current healer iteration is far from a success.

[ItMe]

First we would need to define "success" in terms of healer design.

[GrimGale]

First we would need to define "success" in terms of healer design.

I'd say in this case, both an increase in the proportion of players that favour this specific role and the overall player satisfaction with the role.

Popularity and Satisfaction?

[Acece]

In terms of popularity, I don't think percentage of the healer pop relative to Stb has increased. I know a lot of stb healers that either stopped playing or play dps now. Most of healers I know, shb was their first time healing. So maybe the total healer population has increased with the player base, but percentage wise Idk if it's keeping up. I've also noticed when I was looking for a static that there was a lot of statics looking for healers.

In terms of gameplay it's bad lol.

[Semirhage]

I'd say in this case, both an increase in the proportion of players that favour this specific role and the overall player satisfaction with the role.

Popularity and Satisfaction?

I'm sure there's going to be a surge in healers next expansion, for good or ill. Recency bias will come into play as people try out the new job. I hope Square doesn't take that uptick in healers to necessarily mean they've "fixed" healers.

[AnotherPerson]

Success of good healer design is contingent on a many number of factors, such as:

1. What is the dev's team goal for a healer?
2. How do players like the design goal of the healer?
3. How do players play the healer? Do they play the job as intended, or do they not?
4. Do the players have fun? Are they feeling satisfied?
5. Are fights designed in a way that it's fun to play that content on that job?
6. Recovery - How punishing is it to recover as this job upon a death or mess up? Does it take away the fun besides making it a bit more difficult like every death, or is there something more?
7. Level of improvement - how does the gameplay mechanic change? How does a healer feel improvement when playing better with the same healing toolkit and with an improved toolkit? Is it a noticeable thing?
8. Interaction - How does this job interact with the other jobs? How does it interact with other roles? How does the healer interact with the fight? - This is pretty important and difficult to somewhat manage since there are lot of variables to take into account. Healing affects tanks, overall mitigation for tanks + DPS, fight encounter mechanics, etc.
9. Metrics and feedback - Are there complaints on certain skills? Is there clunky design that hampers gameplay? Is it fixable quickly? Does it affect a large part of gameplay? Is it integral to the job's design? How can it be adjusted? Are there skills that aren't used at all? Are there certain skills that take precedence over others?

A lot of these responses will differ between players who play healers as a side thing, queue in for healer cause 'fast roulettes', are healer mains (does MSQ and most content as a healer), and healers who are taking this role into raid content (career healers) because each kind of player has a separate goal in mind. Some may share similarities, others may not. I'm sure there's more to the list but these are the first couple I can think of on the top of my head.

[Recon1o6]

Itme has hit the nail on the head. First we have to define what counts as success. Second is what are we measuring the success against? Are we comparing the healers as a group to tanks/dps? Are we comparing them to each other?

Its one of those questions where the context is just as important as the response. If you asked "is the healing portion of the healers kit useful?" many would say yes because in a vaccuum the healing kits are powerful and frequent, but if you said "is the healing portion of the healers useful in the majority of gameplay?" more would say no due to the foam bat levels of incoming damage in most of the game making a lot of it wasted hotbar space

To that end you would measure healer design success in four categories
1. mechanical design. Are there any major flaws such as clipping? Are there missing integral actions (ED at launch) Is its healing/damage capabilities significantly lower than others? Players won't play a job if it cant function as we saw with SB whm and monk
2. job satisfaction: are they having fun? what do they like about the job? what do they detest? what are its most frequent complaints? Do they enjoy getting better at the job? how skilled is the average healer?
3. healer pick rates. Are PF and DF groups often waiting for a healer? Are some healers picked over others? How frequent is adventurer in need: healer? Do healers gravitate towards whm over sch etc
4. Design intent vs practice vs player intent: how well do these align? what is it the players want? what are the design goals of the developers with healers, do they align? Does the job design and the battle design match up? Does the way players play healers work with or against the battle design?

Currently, 2, 3 and 4 responses would be very uncomfortable reading for any self respecting developer. 1 is currently sitting at "not terrible but not good either" up from "partly unplayable" at launch. from 1* to 3* out of 5.

On 2- The internet has been flooded with healer complaints about 1 button dps, ease of healing, low incoming damage, homogenisation and a general frustration at being 4th class players (even dol/h got better treatment) Most new healers arent incentivised to improve, veterans have actively discouraged new players from healing. Suffice to say its a 1 or 2* out of 5

On 3- Shield healers were as rare as monk players for most of the expansion. Healers are frequently in need, particularly at peak times and many parties throughout shb needed healers for a while. Bozja allevated some of it with healer exclusive stuff like seraph strike and lost death/repose combo. Veterans have left in droves, New players either dont finish levelling it due to the lack of reward or do and feed into the lower skill ceiling/floor problem plaguing shb. Its plausible more healers would stop were it not for the mandatory 1 healer per light party ratio. 2* out of 5

On 4- its well known that the developers deliberately skipped over the healer slide pre shb. For something like that to happen coupled with what was revealed, is extremely damning. It is known the job design team and battle design team do not interact as much as the playerbase believes they should with e7s being a prime example for healers. Players are spending about 2/3 to 80% of their actions on their few damage buttons, despite the stated job team's goals to make players heal more. Furthermore this came from a false perception that ast/sch were spending more time dpsing and leaving all the healing to the whm. Logs and gameplay proved in sb this was not the case as ast/sch could weave their damage and healing between their gcds whereas whm could not and was weak anyway.
One of the few postives is most healers find the healing portions of the kit solid, but on the other hand the battle design team does not make use of it

Overall, shb healer design has been pretty poor and in many areas, the worst it has ever been.

-healers have lots of consistent complaints
-healer pick rates are low
-huge disconnect between players, job designers and battle designers
-healer satisfaction is at an all time low