Results of a reddit job satisfaction poll

[SamRF]

2700 entries might be enough if you randomly sampled players from all across the active playerbase.

An opt-in poll inherently skews the results because people with stronger opinions are more likely to respond, and you can't say it represents the entire audience of the game just by the numbers. 2700 responses from mostly hardcore players will yield a very different set of responses to 2700 casual players who've never ventured into EX trials.

You can only make statistical claims like this if the data was sampled pseudo randomly from all over the playerbase, as it stands it all comes from r/ffxiv, thats like asking people what their favourite fast food restraunt is while standing outside McDonalds. You also need to take validity into account, you say that its unsurprising that the results agree with sentiment here, but when the most common reason is place in the meta it seems like theyre agreeing for entirely different reasons. Without more statistics to back it up this survey is credible to give you an accurate view of the views of r/ffxiv users, not the playerbase as a whole.

I mean, I've literally said in that post that it's only truly the case if you have a random sample. However, as I've also said, true random sampling is usually impossible. This doesn't mean that those statistical measurements won't be used or taken into consideration (they would be useless if it only gave meaningful information for true random sample). It's not accurate but gives you an idea of what the true accuracy would be when you afterwards take into account how representative the survey entries are.

It's true you need more surveys to confirm this and some results may be up to interpretation but 2700 entries is still quite a lot, imo you can't say the results are meaningless with such large amount of entries assuming vast majority of those were indeed from active FFXIV players (which is a fairly safe assumption to make). Consider that this survey reached more people from the population than many actual (scientific) researchers (any topic) ever hope to be able to reach for their study. It's more than valid material to work with and perhaps right now the best source of feedback for the developers considering the alternative feedback mediums. (It's much better information than the usual feedback on these forums with a thread with perhaps 100 likes or a couple users going back and forth on an issue.)

The fact that it's by far more hardcore players you reach instead of casuals isn't that problematic imo, because casuals likely don't care nearly as much about job satisfaction as those "hardcore" who spend much more time in the game.