Inaccurate Percentages

[Sarnsereg]

I'm starting to wonder if we're being given a false sense of hope on things. I'm not kidding. last night I failed to get HQ on like 5 items where I had 87-98% chance of HQ. I would then turn around and get an HQ on an item I had all of 13% chance on .


maybe someone knows. but does the steady hand only give you you 20% of what the skill already is?

IE... hasty touch is 50%. you get 20% more. well 20% of 50% Is only 10%. so if you use strady hand plus hasty touch you would only have a 60% success rate? I mean. otherwise you would have a 70%.

my math classes would tell me that generally speaking 70% would mean that you succeed around 3 out of 4 attempts.....and I know for a fact that is not true. I consistently will use this and fail 7-10 out of 12 attempts.

[Lemon8or]

Sarnsereg, no it gives you additively. Have you ever seen a Standard Touch fail with Steady Hand or a Basic Touch fail with Steady Hand II? They both add to 100%. It's all RNG really. I've had times where I would not fail any 80% Hasty Touch in 10 minutes straight. The assumption that a 70% chance should give a 70% return in a large number of samples is a fallacy.

[zdub303]

Pretty sure this game just has a horrible random number generator.

[Kakure]

People don't understand math but think they do, news at 11:00!

We are plagued by an enormous number of psychological biases that distort our perception of the world around us.

We are more highly attuned to loss (the times you fail with a 98% chance of success) than gain (the times you succeed with a 2% success chance). We have a tendency to attribute our successes to positive personal characteristics and our failures to external factors; I succeed because I am a good crafter and use my abilities optimally, but I fail because I am unlucky. We vastly overestimate our own capabilities and, more importantly, the accuracy of our own judgment. Something like 90% of drivers rate their own driving as above average, for instance.

(continued)

[Kakure]

When judging causality, we tend to place too much weight on anecdote and personal observation and far too little on objective data.

We are terrible - TERRIBLE - at estimating percentages and even worse at identifying randomness. Human beings excel at finding patterns and that skews our perception of randomness.

Notably, we have evidence from a diversity of fields suggesting that the worse people are at something, the more skewed their perception of their own abilities tends to be. Those who perform the worst on measures of multitasking are the ones who most overestimate their own capacity to multitask. People who perform the worst on math tests are the ones most overestimate their own math abilities.

When dealing with percentage and chance, a good rule of thumb is this:

If the percentage chance displayed on the screen doesn't seem right to you, it is because your perception is inaccurate, not the algorithm.

[CianaIezuborn]

When dealing with percentage and chance, a good rule of thumb is this:

If the percentage chance displayed on the screen doesn't seem right to you, it is because your perception is inaccurate, not the algorithm.

I <3 you can I have your man babies?

[CianaIezuborn]

Warning, dirty statistics that will probably make real statisticians cringe.

However, I also know that over a large sample, statistics should average out. Over a hundred items, if you have a 20% success rate, you should get 20 (plus or minus 2, say) items that you've hit that 20 percent on.

It's sufficiently large, with sufficiently being the key word. It will also never average out, it'll approach the average as the amount of repetitions approaches infinity.

100 repetitions with 20% probability of success:
Expected mean(µ) = 20
Standard deviation(σ) = 4

So really plus or minus 8 (two standard deviations) is plenty inside the bounds of the norm, so you're really looking at 12-28 out of 100 before it's out of the norm. Even then, something out of that norm still has a ~1 in 20 chance of occurring so really you shouldn't be two concerned unless you're 3 standard deviations out of expected mean which would be less than 8 or greater than 32 in 100 attempts.

That's about 6 times the variance you're expecting to see, which is a big issue with people thinking about what they should get when it comes to repetition of chance. That and 100 repetitions can't really be considered a large sample. Consider this, the range within 3 standard deviations (32-8 = 24) is 24% of your domain of 100 repetitions.

For 1000 repetitions with 20% probability:
Expected mean(µ) = 200
Standard deviation(σ) = 12.65
Low end of third deviation = 200 - (3*12.65) = ~162
High end of third deviation = 200 + (3*12.65) = ~238
Range = 238-162 = 76
% of third deviation over domain = 76/1000 = 7.6%

I guess I could have compared the expected mean to the standard deviation, same thing would have happened. As the repetitions increase, the ratio of deviation/expected decreases. This implies that as the repetitions approach infinity that the ratio will approach 0, which covers the whole with enough rolls idea reality will approach theory law that I can't remember the name of.

Finally, what most people mistake as proof that percents/rng/whatever is off is runs in successes/failures. People find it odd that they got 5 20% successes in a row or 5 80% failures in a row. Considering a large amount of repetitions, it's actually STRANGER for runs to NOT happen. I'm not going into that math, cause it makes my brain hurt to calculate.

tldr:
Reasonable variance on 100 rolls on a 20% chance is +- 12. Your reasonable variance is also covering 24% of your entire domain. Better number for a large sample before even beginning to worry about any variance would be about 10000, which would cause your reasonable variance to cover 2.4% of your domain.

[O-Deka-K]

tldr:
Reasonable variance on 100 rolls on a 20% chance is +- 12. Your reasonable variance is also covering 24% of your entire domain. Better number for a large sample before even beginning to worry about any variance would be about 10000, which would cause your reasonable variance to cover 2.4% of your domain.

I agree wholeheartedly with this post.

Most people seem to think along the lines of the OP, that +/- 2 is reasonable over 100 tries. This is understandable. You learn about percentages in math class, but you generally don't get a feeling about what the variance is really like.

After looking at tables upon tables of randomly generated numbers and percentages at my job, I know it's much worse than that. I agree that +/- 12 over 100 tries is way more realistic. As you say, 100 is not a large sample size at all. Ten thousand is a much better number.

[Ed_N_Ants]

You are aware that computers are not capable of coming up with random numbers... Nothing a computer does is random and there is some kind of method to come up with the "randomness"... There are different ways to convey this such as time, paths taken or algorithms but never are they random, they only appear to be...

[CianaIezuborn]

You are aware that computers are not capable of coming up with random numbers... Nothing a computer does is random and there is some kind of method to come up with the "randomness"... There are different ways to convey this such as time, paths taken or algorithms but never are they random, they only appear to be...

Yes, though in the year 2013 freely available psuedo-RNG are more than accurate enough that even their inaccuracy should disappear over sufficient rolls. The danger of a "bad" RNG algorithm is usually either predictability or lack/abundance of runs, the first of which is only evident when you're trying to predict the next number in sequence and the last disappears over large numbers.

Where the errors are these days is usually in the seeding of the RNG, this is a logical error in the programming and usually represents itself as pulling identical numbers/sequences in a row. I have no idea how they seed RNG in an MMO, I'm sure there are multiple strategies out there. My initial thought is that you would attach the RNG object either the character/npc or off of the zone the character/npc is in at the start of login/spawn/server boot and seed it with some hash of the current time. Who knows though.

Edit:
Maybe current time + some ID number on the char/npc/zone hashed? Now I'm interested in their seeding strategy for multi user interfaces, damn you science!