Quote Originally Posted by Yandere-chan View Post
That is an improvement, but since logs are anonymous and this data is from all logs posted, it is neither accurate enough to assess how the whole playerbase is doing nor how players of equal skill do against one another. E.g. If there were 200 Samurai logs posted and 100 Dragoon logs posted, and half of the Samurai logs were of poor optimization and only 30 of the Dragoon logs were of similar poor performance, it would make Samurai look worse than it is. Since there is no curation in FF logs for this by its anonymous upload system and these results are just taking in account all logs posted, that alone is more than enough to make the results unreliable.

As much of a pain it might be, you'd need to individually test each class with all other classes being the same in the test groups, and with the same players for the other classes present, and the players of each comparative class being compared against one another would need to be of similar skill (there would need to be a good way to assess this such as player's performance vs. theoretical max performance). You'd then, with this dedicated team, in both variations, do all end game content multiple times, and then take the avg. of those results. That is as close to accurate as we could get as players. Square likely has access to tools to get even more accurate results though, if they would take the time to test things on their end (it's possible that they have).
That isn't how statistics work. Once you reach a large enough sample size, it doesn't matter. The average will more or less remain the same. Since we have several thousand uploads for every job, this threshold has already been met. What you see on FFlogs is roughly accurate representation of each jobs' output. rDPS is far from a perfect metric but remains efficient. Additionally, FFlogs allows you to partition based on percentile. Therefore, a 95% Dragoon will only be compared against a 95% Bard. And in every single instance, they will absolutely dominant them.