Inaccurate Percentages

[Kakure]

When judging causality, we tend to place too much weight on anecdote and personal observation and far too little on objective data.

We are terrible - TERRIBLE - at estimating percentages and even worse at identifying randomness. Human beings excel at finding patterns and that skews our perception of randomness.

Notably, we have evidence from a diversity of fields suggesting that the worse people are at something, the more skewed their perception of their own abilities tends to be. Those who perform the worst on measures of multitasking are the ones who most overestimate their own capacity to multitask. People who perform the worst on math tests are the ones most overestimate their own math abilities.

When dealing with percentage and chance, a good rule of thumb is this:

If the percentage chance displayed on the screen doesn't seem right to you, it is because your perception is inaccurate, not the algorithm.

[Laryndra]

I call this this the X-Com factor, if you have ever played X-Com Enemy unknown, you will know exactly what I mean.

Difference is Xcom determines whether or not you fail or succeed before you take the action based on the percentage chance. (To prevent you from being able to reload and save to avoid missing at a crucial moment). This game is random. 90% is 90%, still a 1/10 chance to fail, and you can still roll 1 on a 10 sided dice 5 times in a row if you're unlucky.

[NintenPyjak64]

I'm always not sure if it is just me or if it is really like this in the game. I have the feeling that touches on crafts fail more often when the condition is good compared to normal.

This actually makes sense to me considering if you succeed a good or excellent, you get more quality then if it was normal. Only makes sense that the success rate would be slightly less then norm.

[Kakure]

I call this this the X-Com factor, if you have ever played X-Com Enemy unknown, you will know exactly what I mean.

X-COM uses random numbers just like any other game.

The game saves your seed, or maybe a sequence of random numbers pre-generated when you start a new game or load a new zone or whatever, to prevent you from cheating by reloading a saved game to get different combat results. That is intentional and in no way is it evidence that the numbers are anything less than random.

[CianaIezuborn]

Warning, dirty statistics that will probably make real statisticians cringe.

However, I also know that over a large sample, statistics should average out. Over a hundred items, if you have a 20% success rate, you should get 20 (plus or minus 2, say) items that you've hit that 20 percent on.

It's sufficiently large, with sufficiently being the key word. It will also never average out, it'll approach the average as the amount of repetitions approaches infinity.

100 repetitions with 20% probability of success:
Expected mean(µ) = 20
Standard deviation(σ) = 4

So really plus or minus 8 (two standard deviations) is plenty inside the bounds of the norm, so you're really looking at 12-28 out of 100 before it's out of the norm. Even then, something out of that norm still has a ~1 in 20 chance of occurring so really you shouldn't be two concerned unless you're 3 standard deviations out of expected mean which would be less than 8 or greater than 32 in 100 attempts.

That's about 6 times the variance you're expecting to see, which is a big issue with people thinking about what they should get when it comes to repetition of chance. That and 100 repetitions can't really be considered a large sample. Consider this, the range within 3 standard deviations (32-8 = 24) is 24% of your domain of 100 repetitions.

For 1000 repetitions with 20% probability:
Expected mean(µ) = 200
Standard deviation(σ) = 12.65
Low end of third deviation = 200 - (3*12.65) = ~162
High end of third deviation = 200 + (3*12.65) = ~238
Range = 238-162 = 76
% of third deviation over domain = 76/1000 = 7.6%

I guess I could have compared the expected mean to the standard deviation, same thing would have happened. As the repetitions increase, the ratio of deviation/expected decreases. This implies that as the repetitions approach infinity that the ratio will approach 0, which covers the whole with enough rolls idea reality will approach theory law that I can't remember the name of.

Finally, what most people mistake as proof that percents/rng/whatever is off is runs in successes/failures. People find it odd that they got 5 20% successes in a row or 5 80% failures in a row. Considering a large amount of repetitions, it's actually STRANGER for runs to NOT happen. I'm not going into that math, cause it makes my brain hurt to calculate.

tldr:
Reasonable variance on 100 rolls on a 20% chance is +- 12. Your reasonable variance is also covering 24% of your entire domain. Better number for a large sample before even beginning to worry about any variance would be about 10000, which would cause your reasonable variance to cover 2.4% of your domain.

[CianaIezuborn]

When dealing with percentage and chance, a good rule of thumb is this:

If the percentage chance displayed on the screen doesn't seem right to you, it is because your perception is inaccurate, not the algorithm.

I <3 you can I have your man babies?

[Ed_N_Ants]

You are aware that computers are not capable of coming up with random numbers... Nothing a computer does is random and there is some kind of method to come up with the "randomness"... There are different ways to convey this such as time, paths taken or algorithms but never are they random, they only appear to be...

[O-Deka-K]

tldr:
Reasonable variance on 100 rolls on a 20% chance is +- 12. Your reasonable variance is also covering 24% of your entire domain. Better number for a large sample before even beginning to worry about any variance would be about 10000, which would cause your reasonable variance to cover 2.4% of your domain.

I agree wholeheartedly with this post.

Most people seem to think along the lines of the OP, that +/- 2 is reasonable over 100 tries. This is understandable. You learn about percentages in math class, but you generally don't get a feeling about what the variance is really like.

After looking at tables upon tables of randomly generated numbers and percentages at my job, I know it's much worse than that. I agree that +/- 12 over 100 tries is way more realistic. As you say, 100 is not a large sample size at all. Ten thousand is a much better number.

[CianaIezuborn]

You are aware that computers are not capable of coming up with random numbers... Nothing a computer does is random and there is some kind of method to come up with the "randomness"... There are different ways to convey this such as time, paths taken or algorithms but never are they random, they only appear to be...

Yes, though in the year 2013 freely available psuedo-RNG are more than accurate enough that even their inaccuracy should disappear over sufficient rolls. The danger of a "bad" RNG algorithm is usually either predictability or lack/abundance of runs, the first of which is only evident when you're trying to predict the next number in sequence and the last disappears over large numbers.

Where the errors are these days is usually in the seeding of the RNG, this is a logical error in the programming and usually represents itself as pulling identical numbers/sequences in a row. I have no idea how they seed RNG in an MMO, I'm sure there are multiple strategies out there. My initial thought is that you would attach the RNG object either the character/npc or off of the zone the character/npc is in at the start of login/spawn/server boot and seed it with some hash of the current time. Who knows though.

Edit:
Maybe current time + some ID number on the char/npc/zone hashed? Now I'm interested in their seeding strategy for multi user interfaces, damn you science!

[Ksenia]

I'm always not sure if it is just me or if it is really like this in the game. I have the feeling that touches on crafts fail more often when the condition is good compared to normal.

I've noticed that as well. for the first 18 levels on your launch craft you are stuck with a 70% touch and I hate seeing good in that phase of crafting because 90% of the time it meant the touch would fail.

As for general percentage, if you look in the gathering Achievements you will see 30, 300, 1,000, 2000, 4000 based on tier.

I am speculating but I think our crafting algorithm is tied to that. If you are doing a 41-50 craft and you have a 20% base HQ on NQ material, you can expect to craft 4000 of that item before you see the 20% pan out. I also think they shoved all the failures in to the the front of the pack because there is no chance this side of heavens gate anyone will craft 4000 items in a session. So the first 800 are subject to a much higher failure than 20% because the Algorithm would balance out after that and give you hundreds of HQ items in a row. Of course it makes the Algorithm useless as nobody will craft that much at a time.

tldr: We are being rick rolled.