5.08 Patch Notes up

[ForteNightshade]

Yes, that is how statistics work, unless you are throwing accuracy out the window. Anonymous unregulated logs with no control does not equate to reliable test data for class adjustments, not even to a remotely reliable extent. It is not a matter of sample size when the fundamental system is flawed. Any person at any time could purposely upload 100's of extremely poor logs of say, Dragoon, and drastically shift the avg. It's insanity to think a full average of logs from such a system acts as a basis for reliable class assessment. But no one even needs to do this. There can just naturally be a happenstance of an unproportionate size of terrible logs for various roles of those who choose to upload. Massing together averages from all logs from FFlogs will never be a reliable source for objective data on where classes compare against one another. To think otherwise is willful ignorance and a credence to the saying of there being lies, strong lies, and then statistics.

Even if you were going to use FFlogs to try to get an idea of where classes compare against one another, you'd need to spend a long time curating the logs selected, which probably shouldn't be done by just one person (or by a group of like-minded people) anyways due to the concern of personal (or group) bias affecting judgement during the curation process. Honestly though, you'd end up with a far more accurate result if you planned a test from the ground up as described in my original post. What was presented, however, was ignorant at best, lazy at worse, and purposely misleading at worst.

... except they won't shift the average when Dragoon has over ten thousand uploads. A few hundred is throwing a water in a river. It would take several thousand to even move the needle, which is so ridiculously improbable it isn't worth discussing. What you're positing would require a massive coordinated effort to intentionally sabotage logs. Such an endeavor would be very easily noticeable and dealt with long before it did any damage. Put simply, the scenario you're describing just isn't going to happen. Now that doesn't make FFlogs the end all be all, but dismissing it outright or even implying the numbers aren't a decent estimate is just plain ignorance.

[Yandere-chan]

... except they won't shift the average when Dragoon has over ten thousand uploads. A few hundred is throwing a water in a river. It would take several thousand to even move the needle, which is so ridiculously improbable it isn't worth discussing. What you're positing would require a massive coordinated effort to intentionally sabotage logs. Such an endeavor would be very easily noticeable and dealt with long before it did any damage. Put simply, the scenario you're describing just isn't going to happen. Now that doesn't make FFlogs the end all be all, but dismissing it outright or even implying the numbers aren't a decent estimate is just plain ignorance.

Um, no. I said 100's. So, let's say 500. If you have 10,000 logs that avg at say 13k DPS, and you add even 500 logs of say 100 DPS, you will knock it down to around 12385 avg damage. But, as I said, this isn't the point, this doesn't need to be done. This just happens naturally by the nature of random uploads with no control in place. You literally have no basis to even determine if a log being submitted is way below what should theoretically be possible by competent play or not. It's literally just random logs by random people all averaged together. That is NOT accurate, at all. Not even a smidgen of accuracy exists in that. As I said, you could take the time to curate specific logs from the site, but the curation process would need to be very solid, but that even that would fall short of a proper test. To just average it all together though is just folly. It simply isn't a matter of sample size, as I said, the fundamental method that data is being collected by is flawed for the purpose it is being used for in that table.

Edit: It isn't actually as bad as I thought it was because the table is using the 95% percentile at least. I still think that isn't accurate enough, but that is much better than just all logs uploaded on the site like I had misread it as.

[Zyneste]

Um, no. I said 100's. So, let's say 500. If you have 10,000 logs that avg at say 13k DPS, and you add even 500 logs of say 100 DPS, you will knock it down to around 12385 avg damage. But, as I said, this isn't the point, this doesn't need to be done. This just happens naturally by the nature of random uploads with no control in place. You literally have no basis to even determine if a log being submitted is way below what should theoretically be possible by competent play or not. It's literally just random logs by random people all averaged together. That is NOT accurate, at all. Not even a smidgen of accuracy exists in that. As I said, you could take the time to curate specific logs from the site, but the curation process would need to be very solid, but that even that would fall short of a proper test. To just average it all together though is just folly. It simply isn't a matter of sample size, as I said, the fundamental method that data is being collected by is flawed for the purpose it is being used for in that table.

People performing poorly is what affects the percentiles, the skill floor. Deaths out of the control of the Dragoon are the only thing that would skew the accuracy and even then, those sort of deaths will affect all jobs.

If Dragoons have a difficult rotation during certain encounters (or are generally not understanding their class), it will be reflected in the percentiles and it will be easy to see the gap in the skill ceiling amongst players. "Job difficulty will affect performance" or something along those lines. This isn't really bias, this is what FFLOGs is meant to measure. How well each job does while doing mechanics. Dragoons as an example of course.

[Yandere-chan]

People performing poorly is what affects the percentiles, the skill floor. Deaths out of the control of the Dragoon are the only thing that would skew the accuracy and even then, those sort of deaths will affect all jobs.

If Dragoons have a difficult rotation during certain encounters, it will be reflected in the percentiles and it will be easy to see the gap in the skill ceiling amongst players. This isn't really bias, this is what FFLOGs is meant to measure. How well each job does while doing mechanics. Dragoons as an example of course.

I don't think you understand what I'm referring to. Someone made a table that takes every single log on FFLogs and just averages the rdps together to form a basis for how classes compare against one another. Percentiles on FFlogs isn't in the discussion as it has nothing to do with the table. Percentiles just show you how good you are doing relative to others of your class vs. everyone else by log info. Uploading a ton of poor results would just end up making everyone else look slightly better (considering how low 100 dps is, pretty much any even casual player would move up from such an influx). But this doesn't really matter with the point that just averaging all logs ever together on the site isn't a good way to determine where classes compare against one another. You'd be way better off curating which logs you are averaging to avoid absurdly bad logs that make no sense. Of course, an actual test that is very well organized and controlled is going to be more reliable than even that.

Edit: Wait, nvm. It isn't quite as bad as I thought. It is only 95% percentile. While I still don't think that's nearly accurate enough since there isn't a good way to determine if the 95%+ percentile's of each role is of equal standing to one another in what classes are capable of with what's on the site, it isn't as bad as just all logs being averaged. I would take the table results with a grain of salt at best. You'd be better off having a more detailed selection for which logs to compare than just the 95% across the board if you were to use the site's recorded logs.