Quote Originally Posted by Koros View Post
Game companies are NOT good at interpreting those numbers.

I highly doubt SE is any different.

The task that they have is monumental. Engagement with a piece of content Y is the result of multiple factors X (X1, X2, ...). But they quite literally cannot observe most of those factors while they probably harvest a ton of data most of which is irrelevant. Most game companies would not have a single staff who understands how to do causal inference on high dimensional time series data. If they do they're working at Amazon for 300k and I've literally never seen game companies hire actual data scientists or statisticians of this calibre.
Forced to agree.

Similarly, what you will very often see with small game devs who don't employ anyone who knows how to competently analyze metrics are circular feedback loops which create high-pressure events that coerce or funnel players into a specific activity that they hate, but feel like they have to put up with, in order to get a reward - they then point at the predictably-high engagement numbers as "proof" that players like that activity.