Wesk Alber's poll on Jobs

[Renathras]

Video with results here: https://www.youtube.com/watch?v=W5V2xhS8GKs

Note his sample size is around 4,000, which is ROUGHLY 20x what Ty's poll was. It isn't based on a 1-10 scale, but seems rather to be people saying which Jobs were their favorite.

Of note here I just wanted to pull from the healer section:

WHM: 29%
SCH: 19%
AST: 21%
SGE: 31%

He notes the comments have a fairly even split, but in terms of popularity, SGE > WHM > AST > SCH, which is interestingly more or less the opposite order people on these forums tend to rate the enjoyability/favorite-ness of the Jobs. (Not exact, but roughly), with the one exception of SGE being fairly enjoyable to everyone.

I'm not posting this as some kind of "gotcha".

I'm more posting this to point out the importance of sample. Who you poll is what's going to determine the outcome of polls.

Looking at the poll done here, it was basically SGE > SCH > WHM ~= AST. His poll was SGE > WHM > AST > SCH, with a larger sample size, but probably a different type of player. As he says when talking about Tanks (and why he was surprised WAR wasn't higher), he thinks his audience is more casual players.

(Incidentally, SMN also did FAR better under his poll)

It is true the polls were also asking for different things, but as I commented on Ty's BLM result, because he didn't limit it in any way (no requirement people answer for all Jobs OR limitation on their favorite), we ended up with cases of BLM being super highly rated, despite tons of people hating it, because the only people who rated it were people that love it for the most part. And as Alber points out, some Jobs (BLM in particular) have really stark love/hate relationships, meaning if the hate people aren't bothering with it, then it's going to have an inflated result (BRD and SCH were two others he pointed out).

.

The one big takeaway for healers - besides the importance of who is taking your poll - is that both groups seem to like SGE.

[ty_taurus]

It is true the polls were also asking for different things, but as I commented on Ty's BLM result, because he didn't limit it in any way (no requirement people answer for all Jobs OR limitation on their favorite), we ended up with cases of BLM being super highly rated, despite tons of people hating it, because the only people who rated it were people that love it for the most part.

I have a question. Let's say, for the sake of this example, that I hate coffee. In reality, I love coffee, but let's pretend that I despise it: the taste, the smell, the effect it has on my body... everything. Now, let's say you love coffee and everything about it. If you were to ask me, while sitting at a café, what I thought of the in-house roast, and I responded with something like "It's disgusting. Don't get it." How useful is that information to you?

My point is, why should the input of someone who does not like coffee matter when discussing the quality of a particular brew? Or translating that to Black Mage, why should the opinion of someone who is not interested in turret-style caster gameplay be considered when discussing the overall quality of Black Mage as a job? How would a designer use that feedback to improve the quality of the job for the players that are looking for a turret-style caster?

Meanwhile, Summoner bombed in my poll. Why? Summoner experienced a complete overall in Endwalker, so first and foremost, there was an already established playstyle that responders were comparing the current design to, something that Black Mage hasn't experienced, but it can also be said that not all players feel new Summoner delivers on the expected playstyle of the job, which can be seen with many of the comments on what people wanted to see changed with the job. Keep in mind, Monk also experienced an overhaul in Endwalker, and while its numbers weren't the best amongst my findings, Monk had a fairly average overall review despite also being in the same position as Summoner. So you can't say Summoner's scores were sabotaged by salty former Summoner players. And taking a look at the video example, we already know that few players play Black Mage, yet Black Mage still surpasses Summoner despite Summoner having a much higher population of players.

As for the healer findings... Does knowing Sage is marginally more preferred amongst healers actually tell you anything? How would that information help a designer trying to improve the quality of healer gameplay? Does it actually tell you anything about how well-designed the community feels the healers are?

For the record, having such a large sample size is great, and it's nice to see that much passion poured into his video. But I also don't feel like his results really communicate anything substantial. Sure you know what jobs are the most preferred amongst each role, but just because someone submits which healer is their favorite doesn't necessarily mean they like playing as a healer, and the same is true for all the other roles as well.

[Renathras]

I have a question. Let's say, for the sake of this example, that I hate coffee. In reality, I love coffee, but let's pretend that I despise it: the taste, the smell, the effect it has on my body... everything. Now, let's say you love coffee and everything about it. If you were to ask me, while sitting at a café, what I thought of the in-house roast, and I responded with something like "It's disgusting. Don't get it." How useful is that information to you?

Very.

Consider the fact that a lot of players don't like current Healers. Yet they rated current Healers, giving them artificially low marks. A lot of players who do not play SMN hate current SMN. It is the single lowest rated Job in your poll, despite possibly being the most played DPS and second or third most played Job in the game. And while I KNOW you want to say "people play it but don't like it" - I play it and love it.

Meanwhile, BLM came out smelling like a rose despite the fact that it's long been one of the least played Jobs in the game, and outside of high end raiding, is still one of the least played Jobs in the game. It also got, I believe, among the least (if not the least) responses in your poll. Meaning most people who don't like BLM didn't brigade its category giving it low marks - something that likely DID happen with SMN and probably the Healers - giving it an artificially higher rank. Even among this forum as respondents, BLM would probably have gotten lower had everyone voted for it, and it seems to be the least liked Job in the game. The fact your result says its the most loved means your poll isn't representing reality. All your poll captured on BLM is "Among BLM mains/enthusiasts, it's the most loved", since those were probably the people voting for it. But we don't even know THAT for sure, making the result pretty close to meaningless.

(Personally, I think that OF PEOPLE WHO LIKE BLM, they really like it, meaning the design is good since it appeals to the people that enjoy that kind of design. But this is PROBABLY also true of other Jobs, like SMN, which your poll seems not to reflect at all.)

If players had to rate all Jobs, then we'd see a lot of the "look at how much people love all these non-healer/non-SMN Jobs!" number reflect...well...reality, and they'd probably have lower marks comparatively, which gives us a better point of comparison about what people do and don't like.

Conversely, if players only rated ONE Job, whatever their main/favorite is, then we'd get a far better look at how much the Jobs appeal to the people that play them most/that they appeal to most; which is far more useful for Job design. Taking your example, it makes more sense to only ask coffee drinkers to rate a new coffee than it does to have everyone rate it. Using your example above, you shouldn't have allowed people to rate Jobs they don't play, as that's like me asking you how much you like the in-house roast and you saying "It's disgusting. Don't get it."

In other words, your own argument there is valid for attacking your poll methodology.

SMN bombed in your poll because your poll didn't ask people to only rate the Jobs they main (where SMN mains would probably rate it higher than people who do not play SMN because they hate it), and only bombed RELATIVE TO the other Jobs because you wise didn't have as many people poll-bombing BLM or other similar Jobs.

In short, unless you require people to vote on all the Jobs OR only allow votes on people's Main, it means you can't use the numbers compared to others, which kills the entire point of the poll. "SMN got a 3 while BLM got an 8" only tells you "SMN got a 3 among those who voted for it - people who love it and hate it, play it and refuse to play it - BLM got a 8 among those who voted for it - people who love it and hate it, play it and refuse to play it...but we don't know how much of the vote for each were the people that love and play it and those who hate and don't play it."

I mean, you get people like me who hate BLM and would rate it 1s across the board who...just didn't vote in the BLM category. So you don't actually have a comparison between the Jobs in your poll since it's different groups of people voting on the Jobs as opposed to everyone voting on them all OR only people who they already appeal to. The former would be useful for comparison between Jobs, the latter for seeing what appeals and doesn't to people who actually play Jobs and what changes people who actually play Jobs - the people changes and design should be targeted towards - care about.

Like imagine if the bulk of people rating BLM were people who hated BLM and refused to play it because they don't like cast bars.

It's like how I rarely make suggestions for AST here, because I recognize AST appeals to people who aren't me, and THEY are the ones who SE should be listening to for design suggestions, not people like me who don't like the Job and its mechanics anyway. I should be the one they listen to for stuff like WHM, since that's something I've played for close to a decade and love very much. I keep to that rule so strongly because I feel strongly on the point.

.

As to the last point on SGE:

It's kind of interesting since you probably have to dig into the why. It seems the most popular Job of the Healers in both your poll and his. Alber presents it as "a better SCH", which it arguably is and I think some of the comments responding to your poll said as much, too. That people seem to want a Job that's "smooth". It doesn't have a lot of the clunk that SCH does, and it doesn't have things like Energy Drain fighting over Aetherflow stacks. It has good MP regeneration, decent personal damage, a massive amount of mitigation (which encounter design currently prizes very highly), lots of mobility (it is THE most mobile Healer), flashy animations, and is a Barrier Healer that can compete in raw healing with either one of the Pure Healers.

Questions would have to be asked about the specifics to see what people DO like about it - something your poll did and I'm sure his responses did as well (I think it was a poll on his Twitter, so there's probably a lot of replies made in the comments) - but I think the point that's valuable is that it's the one Healer that seems to be most appealing to both groups of players. The casual types and more general audience like it and the hardcore types that frequent internet forums like it...passingly.

.

I also think it's fair that if people are voting on their favorite, they PROBABLY like it. They may not love it, that's the distinction you could probably go with, but they probably like it. People weren't forced, as far as I can tell, to vote on the roles they didn't like, as the polls were split by role and someone could simply not vote in the roles they don't care for any of the options of. Note also that each poll has the total number of votes on the top, and each category has a different number of respondents. Meaning if people didn't like ANY of the Jobs in a category, at least some of those people simply didn't vote in that category.

The rest is kind of your interpretation, but we have no support for that interpretation unless people were saying "I don't like any Jobs in this role, but this one is my least hated".

I think it's fair that people probably like them unless you have solid reason to indicate they do not.

[ty_taurus]

Very.

Consider the fact that a lot of players don't like current Healers. Yet they rated current Healers, giving them artificially low marks. A lot of players who do not play SMN hate current SMN. It is the single lowest rated Job in your poll, despite possibly being the most played DPS and second or third most played Job in the game. And while I KNOW you want to say "people play it but don't like it" - I play it and love it.

What is the context of Summoner's population? Why does it have higher numbers than Black Mage? Sure, there are people who do play the job, but how many of that population leveled it for the purposes of clearing the caster role quest? how many leveled it for the purposes of soloing Deep Dungeons? Summoner is very strong for Heaven on High, for example. How many leveled it simply to try it? How many leveled it because they play Scholar and have Summoner leveled as a consequence? How many people have leveled it for the convenience of having it?

More importantly, if FFXIV were like WoW in that you could not swap jobs. You picked one job and that was your only job. How popular would Summoner actually be? It's very likely that Black Mage's numbers probably wouldn't change drastically, because the people that play it have leveled it because they played it, but I don't think that's true for Summoner. I think a major chunk of its population have it leveled because it's convenient. You get 2 jobs for the cost of 1. You can use it to clear your role quests. You can use it to solo easily if you normally main tank or healer and want something faster. There are many reasons to level and have Summoner in your back pocket regardless of whether or not you actively play and/or enjoy the job, and leveling it does not inherently mean you do enjoy it.

I've said before, I count on the census for every job in the game. I have every job at 90, and I have every Manderville weapon. Of those, I actively use 2-3. I may not be the norm, but it's considerably easier for someone who has 5 jobs leveled, one for each role, to stay on top of having a decent weapon now more than ever and count for the census regardless of whether they actively play all 5 of those, or they just play 1 and use the other 4 as needed. Of the casters, Summoner is undoubtedly the most convenient to use for those purposes. The census doesn't tell you who mains what. It just tells you who has what.

I also think it's fair that if people are voting on their favorite, they PROBABLY like it.

My favorite tank is Gunbreaker. I've never set foot in a single instance as Gunbreaker, never want to play Gunbreaker, and have a negative amount of interest in playing as a tank at all. Does Gunbreaker being my favorite tank matter, in that case? Would you say I "like" Gunbreaker? Or is it my favorite because I don't actually play that role, and of the options, it's the one I think is the coolest in concept? And that my general lack of familiarity with tanks means what I like says absolutely nothing about the job's design?

I'm not saying that's every player. But if everyone is required to pick their favorite of each role, any votes for jobs in a role that someone doesn't actually play aren't helpful and don't provide any meaningful information other than "I guess people like the style of this job" which is probably why Astro rated slightly higher than Scholar. People think Astro looks cool, so many players who have never even touched healers may have voted for it simply due to aesthetic. And that's also my guess as to why Sage is the highest. White Mage and Sage are higher than Scholar and Astro because Scholar and Astro suffer from pain points--pet mechanics and a kit that fights with itself for Scholar, and constant single-target shuffling and double weaving for Astro.

All that the healer data tells me is that of the healers, Astro and Sage look the coolest, but Sage is the one without a pain point and thus doesn't take a score hit due to actual healer players rating Astro lower as a result of its pain points.

[Renathras]

What is the context of Summoner's population? Why does it have higher numbers than Black Mage?
...
More importantly, if FFXIV were like WoW in that you could not swap jobs. You picked one job and that was your only job. How popular would Summoner actually be?
...
It's very likely that Black Mage's numbers probably wouldn't change drastically,

We don't know, and your poll did nothing to help answer those questions, as it didn't ask any of those questions.

It's PROBABLY got something to do with it being relatively easy to play and that a lot of players enjoy it, but we really don't know for sure.

And we also don't know that BLM's numbers would be the same or not. If players were forced to ONLY have one Job, many would probably not pick BLM since players tend to like having options, and BLM is not flexible. You're assuming that people who like BLM still would, but how do we know players don't play BLM who don't like it because they like that it does the most damage and they can have a second Job on the side? That if that side Job option were no longer there, they'd just go with...SMN? Or RDM?

For example, I realized a long time ago I just really like access to heals, so I never pick classes in games that don't have SOME kind of healing or at the very least support buffs. The only Jobs I would possibly pick if we could only have one would be PLD, SMN, RDM, or the Healers. These are also the Jobs I generally main. When I started in ARR, that would have been PLD, WHM, SCH, or SMN. But...these are the same Jobs I picked then, and the same ones I main now. So this isn't an argument that can apply to BLM only, and it works both ways - there are probably people that main BLM because they can swap to other Jobs for other forms of content, and if they didn't have that option, would probably just main RDM instead.

We don't know that, either. You posit it as if it's "very likely", but it's also "very likely" if players were limited to only one Job, there are many BLMs that would have picked a different Job instead. We really don't know, and again, your poll didn't ask that question, so offers no answers.

As someone who doesn't like SMN and doesn't seem to play it (you've not really talked about playing it here that I can recall), you make a lot of assumptions about what people who do like or or who do play it play it for, and what their opinions are on it and why they may or may not be playing it. From forbidden site, as far as I can tell, DNC and SGE are the only Jobs you play (or get snapshots playing), not SMN. So I think it's fair to assume you don't frequently play SMN. Yet you seem to have a lot of reasons that you think people who aren't you play it, other than "They actually like it".

Note, on the other hand, I DO play it and have had snapshots of me playing it end up there. And I actually LIKE it. I don't play it for any of the reasons you listed.

I've said before, I count on the census for every job in the game.

Which census? Lucky Bancho, as far as I can tell, uses only the Job you were on when the census snapshot was taken. It doesn't count players for every Job. The one that does it by ilevel (generally having the Extreme's weapon or better) also only does it for people logged with that weapon equipped, I think.

So I don't think you count for every Job per the LB census. I don't think anyone does, unless he's changed his methodology.

Or were you referring to a different census?

Would you say I "like" Gunbreaker?

Did you vote in the poll saying you like GNB?

I'm going to guess you did not.

(Yes, yes, you didn't vote in his poll at all, but that's kind of my point; you don't know the people who voted or why they did. All that we know is each category got a different number, so it's likely people who don't care for a given role just didn't vote in that role's poll. AT LEAST IN PART.)

I'm not saying that's every player. But if everyone is required to pick their favorite of each role,

In is poll, they were not. Again, different number of results for each poll says that everyone was not required to pick from each one.

which is probably why Astro rated slightly higher than Scholar.

This is probably less likely, actually.

Note where he brings up the prior poll and the SCH number was effectively unchanged. His conclusion is probably correct that SCH players are "ride or die"; that is, those who liked it probably still do. It's number, in fact, was remarkably similar to the last poll. Which makes sense, SCH changed very little from ShB to EW, so if you liked it in ShB, you probably do in EW unless SGE became more your favorite. It seems SCH pulled numbers from AST and to a lesser extent WHM, with AST's numbers crashing as they were cannibalized by SGE.

White Mage and Sage are higher than Scholar and Astro because Scholar and Astro suffer from pain points--pet mechanics and a kit that fights with itself for Scholar, and constant single-target shuffling and double weaving for Astro.

And/or...people actually like WHM/SGE.

It's weird to me how far you bend over backward to insist that people CAN'T or DON'T like (at least not in large part) Jobs that you dislike.

You're an intelligent person. Surely you realize that there are people who like what you do not, and there may be a lot of them. So why are you so insistent that people CAN'T enjoy these things?

All that the healer data tells me is that of the healers, Astro and Sage look the coolest, but Sage is the one without a pain point and thus doesn't take a score hit due to actual healer players rating Astro lower as a result of its pain points.

This is such a strange analysis to me...again, you seem adamant of finding any other answer possible besides "people like it".

[Shurrikhan]

I feel like without a basepoint, an overall job satisfaction metric is not going to give actionable information on the job itself.

Now, that basepoint can come in a few different ways.

If I first got data on what in general appeals to a given player, from within factors limited to what can actually be seen across all qualities of runs/content (e.g., at highest tiers, not in a shitshow, but also without static-level coordination), I could then make some real use of each responder's satisfaction data by looking also at what qualities seem to be covered by each job and then the discrepancies between thereby-expected results and the actual results.

Similarly, if I narrowed my subjects of inquiry to those skilled in multiple jobs within the role but who mained just the one particular job and had their satisfaction ratings on the job at one or more prior points in time, I could get a feel for how changes were since received.

But the opinion on Monk coming from an utterly new player vs. someone who mained it through Heavensward and Stormblood, or someone who plays only Monk vs. someone who plays all jobs at a skilled level, for instance, are likely going to be answering very different questions because they're effectively not just playing the job but also the sort of negative space surrounding that job -- what it had but now lacks, what other jobs lack but it has, or what annoying things other jobs have but it lacks, just to name a few aspects.


By contrast, evaluations of job gameplay changes can get nearer to something useful, but that already narrows the question significantly, reducing conflation.

[Semirhage]

This is such a strange analysis to me...again, you seem adamant of finding any other answer possible besides "people like it".

I mean, pot meet kettle. Any data that shows people dislike healers is either irrelevant anecdata or merely the rabble-rousing of a small insignificant minority who aren't representative of the vast majority of people who just loooooooove current design. Any data that shows people like current job design is correct and that's the only reasonable interpretation.

[Supersnow845]

I mean it’s not hard to predict this data result, amongst the wider community (that includes sylphies and people who don’t like healers but like having an opinion on them) the so called easy healers are going to score better

In a poll on the forums on a space dedicated to people who basically think healers are collectively a mess the shield healers are going to score better because while all the healers are a mess the shield healers at least play to 14 more effectively

If you just asked me to pull a poll result out of nowhere I would have ranked it
Entire community WHM>SGE>AST=SCH
Core actually cares about healer community SGE>SCH>AST>WHM

Which is pretty much exactly what the result was

[Renathras]

...

I do agree basepoints are important, but you can't limit those to "at highest tiers". That's going to give a VERY biased perspective that's also for a limited amount of content and a limited amount of players.

I do think it would be interesting to ask a question like "If this Job is one you play/main a lot and enjoy, what things do you enjoy about it?"

Again, Job design should be focused on those who actually like the Jobs, not the people who don't like a Job. Jobs are naturally going to appeal to some people and not others. BLM does not appeal to me at all...so why should they design it to where I like it? Same with AST which only vaguely appeals to me in terms of buff mechanics and little else.

And this cuts both ways - old SMN should have had changes made to it to make happy the people who liked it. It's why I consistently say new SMN as a new Job makes sense to me, but not having overwritten that one unless it just uniquely appealed to the people who like that one and WAS what they wanted...and for the most part, it was not and that much is clear.

Your last point is also an interesting one. In order to get REALLY useful data, we need to ask...pretty narrow questions. The problem is, if the questions are too narrow, they may be useful but of limited value because of how they're only useful to that specifically narrow thing. But on the other hand, asking with too broad a net doesn't get really useful information other than overall trends, and sometimes not even that, biased by the sample/pool of respondents that you have answering the poll.

I do think it's instructive to compare various polls - in politics, polling aggregates TEND to be more correct because individual polls may have errors, but get enough of them together and you have some errors in one direction cancelled by errors by other polls in the other direction - but even so, it's still more of a "This is interesting", and "This is what they agree on", but still limited.

.

I mean, pot meet kettle.

Of all the things to try and accuse me of hypocrisy on, Semi, this is the worst one possible for you to pick.

I've been EXTREMELY consistent in noting other people like things I do not. Hell, I was even doing it in this very thread talking about BLM and AST. I've been consistent for my posting history here on not proposing changes to AST because of it, with the few times I have proposed such changes pointing this out as well. The ENTIRE PREMISE behind my "4 Healers Model" is BASED ON THIS VERY IDEA.

Note I also have not said that "the vast majority" love current Healer design. In fact, I avoid saying "majority" at all.

This is literally the worst possible attempt of attack you could make on me, and everyone here probably knows it, even if, as I suspect, none will come to my defense on it.

Any data that shows people dislike healers is either irrelevant anecdata or merely the rabble-rousing of a small insignificant minority who aren't representative of the vast majority of people who just loooooooove current design. Any data that shows people like current job design is correct and that's the only reasonable interpretation.[/QUOTE]

I mean it’s not hard to predict this data result, amongst the wider community...

...of people that play the game, I agree. The more straightforward Jobs will score higher because across the Human population, people seem to have a preference for them.

AST scored lower than WHM in the poll conducted here, actually. WHM was also initially higher than SCH then the poll got bombed with lower WHM scores to reduce it, but at least the last time I looked, AST was still the lowest by far.

[ty_taurus]

This is such a strange analysis to me...again, you seem adamant of finding any other answer possible besides "people like it".

Except I didn't say "no one actually likes Summoner or Healers" or anything to that effect. What I'm saying is population data can't just be taken at face value because you have to ask why. And I'm not trying to answer "why," I'm just listing a variety of reasons that can potentially be true for some people. How many people? Who knows? Maybe only 5% of the census data, maybe 45%. Without anything to go off of, there's no way to know. But between Black Mage and Summoner, Black Mage is very much a "ride or die" type of job, whereas Summoner is not. It's a very convenient job to take advantage of regardless of your opinion on it because it's accessible, easy, and it's stylish.

Which census? Lucky Bancho, as far as I can tell, uses only the Job you were on when the census snapshot was taken. It doesn't count players for every Job. The one that does it by ilevel (generally having the Extreme's weapon or better) also only does it for people logged with that weapon equipped, I think.

If that's the case, then that information is even less reliable in my eyes. Who knows what job I was wearing when that was taken. It might be Dancer. It might be Sage. It might even be a Tank if it was taken when I was grinding them up to 90. Who knows? I don't tend to swap back to my main when I log out or anything. I just log out in whatever I'm wearing, which is currently Dancer a lot just because I've only been logging in to do savage with my raid group and then log out, but that's not always the case.

And/or...people actually like WHM/SGE.

It's weird to me how far you bend over backward to insist that people CAN'T or DON'T like (at least not in large part) Jobs that you dislike.

Are you on something? The video you shared showed values out of 100%, which has nothing to do with how many people like current healer design and only shares info on what healers are the most or least liked from those who voted on a healer. I am saying that Scholar and Astro are likely so far below White Mage and Sage because of their pain points. You have seen many a player actively complain about both Scholar and Astro. You cannot deny that plays some effect into their overall popularity. That doesn't mean "You know if Scholar and Astro played more smoothly, it'd be 50% Scholar and 50% Astro cause White Mage and Sage SUCK." It means there are annoying aspects to playing as Scholar and Astro which undoubtedly has some effect on their low popularity.