Interesting Tank Research

[Chrono_Rising]

The study certainly has issues, and I also believe the focus on dark knight/the provocative tone is a weakness of the write up which takes attention away from what was actually found about tanking compositions.

However, I do want to point out a double standard here. If we want to demand peer review quality of work to justify a claim that we don't like, then we also need to require peer review of the statements being made that we do like. This is a process which takes time, as someone who submits work for peer review I would say a short turn around is 6 months to get through the refereeing process, and up to a year to see publication (depending on field). So if we want to hold our judgments of classes for half the life of an expansion before telling the developers and community what we think/critically assess so be it, but realize that applies to all criticism and claims, not just the ones we disagree with. To give an idea of time to produce just that easy statement on total damage taken the time to collect the data, remove party effects, and remove runs using tank stance, having weakness, tank deaths, and other such things which would skew results and then complete an analysis, interpret the results took nearly 60 to 80 hours of people's free time. If you want more nuanced statements about survivability more work and more in-depth work needs to be done, and I expect it will also take a large amount of time.

For a video game hobby, I doubt we are going to find people who will have the time/energy to submit research to a peer review process to justify multiple narrowly defined questions.

On the other hand, if we want to share ideas and improve people's methodology for teasing out these statements with actual justified numbers, then we need to talk and share as a community. Peer review process is going to slow that down and limit the information to a small subset of people for a long time (remember peer review requires the review be carried out by (a) qualified people (in this case requiring some advanced statistics knowledge) and (b) be impartial to the results), and isn't a road we should demand of people. That said, the paper has weaknesses, and one is that it only looks at total damage intake, and not buster/spike damage, which is more important for assessing how well tanks are surviving.