Inaccurate Percentages

I've been watching the reported percentages. I took statistics in college, and have played tabletop RPGs with dice long enough that I understand that sometimes, you roll more 1s on that d20 than you'd expect. However, I also know that over a large sample, statistics should average out. Over a hundred items, if you have a 20% success rate, you should get 20 (plus or minus 2, say) items that you've hit that 20 percent on.

That's not what I'm seeing. There seem to be variables which are unreported, or the game has a really bad random number generator. Basically, if I have less than an 80% chance of success, I have about a 50% rate of actual success.

I'd prefer if the number shown reflected all variables, and actually bore some resemblance to the success rate.

I call this this the X-Com factor, if you have ever played X-Com Enemy unknown, you will know exactly what I mean.

I'm always not sure if it is just me or if it is really like this in the game. I have the feeling that touches on crafts fail more often when the condition is good compared to normal.

Data or it didn't happen. This better not be another one of those "oh, I just eyeballed it" posts.

There is some nasty RNG in this game but I'm not going to 100% say that it's definitely because the percentages are lying. It could be that you're just really unlucky.

Years of playing Pokemon in my youth got me acquainted very quickly with what accuracy % I found acceptable and what would just lead to me yelling BS at the game. I found 80% to be around where I draw the line regardless of what the ability does when it hits.

I'm starting to wonder if we're being given a false sense of hope on things. I'm not kidding. last night I failed to get HQ on like 5 items where I had 87-98% chance of HQ. I would then turn around and get an HQ on an item I had all of 13% chance on .


maybe someone knows. but does the steady hand only give you you 20% of what the skill already is?

IE... hasty touch is 50%. you get 20% more. well 20% of 50% Is only 10%. so if you use strady hand plus hasty touch you would only have a 60% success rate? I mean. otherwise you would have a 70%.

my math classes would tell me that generally speaking 70% would mean that you succeed around 3 out of 4 attempts.....and I know for a fact that is not true. I consistently will use this and fail 7-10 out of 12 attempts.

Sarnsereg, no it gives you additively. Have you ever seen a Standard Touch fail with Steady Hand or a Basic Touch fail with Steady Hand II? They both add to 100%. It's all RNG really. I've had times where I would not fail any 80% Hasty Touch in 10 minutes straight. The assumption that a 70% chance should give a 70% return in a large number of samples is a fallacy.

Pretty sure this game just has a horrible random number generator.

People don't understand math but think they do, news at 11:00!

We are plagued by an enormous number of psychological biases that distort our perception of the world around us.

We are more highly attuned to loss (the times you fail with a 98% chance of success) than gain (the times you succeed with a 2% success chance). We have a tendency to attribute our successes to positive personal characteristics and our failures to external factors; I succeed because I am a good crafter and use my abilities optimally, but I fail because I am unlucky. We vastly overestimate our own capabilities and, more importantly, the accuracy of our own judgment. Something like 90% of drivers rate their own driving as above average, for instance.

(continued)

When judging causality, we tend to place too much weight on anecdote and personal observation and far too little on objective data.

We are terrible - TERRIBLE - at estimating percentages and even worse at identifying randomness. Human beings excel at finding patterns and that skews our perception of randomness.

Notably, we have evidence from a diversity of fields suggesting that the worse people are at something, the more skewed their perception of their own abilities tends to be. Those who perform the worst on measures of multitasking are the ones who most overestimate their own capacity to multitask. People who perform the worst on math tests are the ones most overestimate their own math abilities.

When dealing with percentage and chance, a good rule of thumb is this:

If the percentage chance displayed on the screen doesn't seem right to you, it is because your perception is inaccurate, not the algorithm.

Quote:

---

Originally Posted by ReviiLagoon

I call this this the X-Com factor, if you have ever played X-Com Enemy unknown, you will know exactly what I mean.

---

Difference is Xcom determines whether or not you fail or succeed before you take the action based on the percentage chance. (To prevent you from being able to reload and save to avoid missing at a crucial moment). This game is random. 90% is 90%, still a 1/10 chance to fail, and you can still roll 1 on a 10 sided dice 5 times in a row if you're unlucky.

Quote:

---

Originally Posted by roohan

I'm always not sure if it is just me or if it is really like this in the game. I have the feeling that touches on crafts fail more often when the condition is good compared to normal.

---

This actually makes sense to me considering if you succeed a good or excellent, you get more quality then if it was normal. Only makes sense that the success rate would be slightly less then norm.

Quote:

---

Originally Posted by ReviiLagoon

I call this this the X-Com factor, if you have ever played X-Com Enemy unknown, you will know exactly what I mean.

---

X-COM uses random numbers just like any other game.

The game saves your seed, or maybe a sequence of random numbers pre-generated when you start a new game or load a new zone or whatever, to prevent you from cheating by reloading a saved game to get different combat results. That is intentional and in no way is it evidence that the numbers are anything less than random.

Warning, dirty statistics that will probably make real statisticians cringe.

Quote:

---

Originally Posted by JeniLinsky

However, I also know that over a large sample, statistics should average out. Over a hundred items, if you have a 20% success rate, you should get 20 (plus or minus 2, say) items that you've hit that 20 percent on.

---

It's sufficiently large, with sufficiently being the key word. It will also never average out, it'll approach the average as the amount of repetitions approaches infinity.

100 repetitions with 20% probability of success:
Expected mean(µ) = 20
Standard deviation(σ) = 4

So really plus or minus 8 (two standard deviations) is plenty inside the bounds of the norm, so you're really looking at 12-28 out of 100 before it's out of the norm. Even then, something out of that norm still has a ~1 in 20 chance of occurring so really you shouldn't be two concerned unless you're 3 standard deviations out of expected mean which would be less than 8 or greater than 32 in 100 attempts.

That's about 6 times the variance you're expecting to see, which is a big issue with people thinking about what they should get when it comes to repetition of chance. That and 100 repetitions can't really be considered a large sample. Consider this, the range within 3 standard deviations (32-8 = 24) is 24% of your domain of 100 repetitions.

For 1000 repetitions with 20% probability:
Expected mean(µ) = 200
Standard deviation(σ) = 12.65
Low end of third deviation = 200 - (3*12.65) = ~162
High end of third deviation = 200 + (3*12.65) = ~238
Range = 238-162 = 76
% of third deviation over domain = 76/1000 = 7.6%

I guess I could have compared the expected mean to the standard deviation, same thing would have happened. As the repetitions increase, the ratio of deviation/expected decreases. This implies that as the repetitions approach infinity that the ratio will approach 0, which covers the whole with enough rolls idea reality will approach theory law that I can't remember the name of.

Finally, what most people mistake as proof that percents/rng/whatever is off is runs in successes/failures. People find it odd that they got 5 20% successes in a row or 5 80% failures in a row. Considering a large amount of repetitions, it's actually STRANGER for runs to NOT happen. I'm not going into that math, cause it makes my brain hurt to calculate.

tldr:
Reasonable variance on 100 rolls on a 20% chance is +- 12. Your reasonable variance is also covering 24% of your entire domain. Better number for a large sample before even beginning to worry about any variance would be about 10000, which would cause your reasonable variance to cover 2.4% of your domain.

Quote:

---

Originally Posted by Kakure

When dealing with percentage and chance, a good rule of thumb is this:

If the percentage chance displayed on the screen doesn't seem right to you, it is because your perception is inaccurate, not the algorithm.

---

I <3 you can I have your man babies?

You are aware that computers are not capable of coming up with random numbers... Nothing a computer does is random and there is some kind of method to come up with the "randomness"... There are different ways to convey this such as time, paths taken or algorithms but never are they random, they only appear to be...

Quote:

---

Originally Posted by CianaIezuborn

tldr:
Reasonable variance on 100 rolls on a 20% chance is +- 12. Your reasonable variance is also covering 24% of your entire domain. Better number for a large sample before even beginning to worry about any variance would be about 10000, which would cause your reasonable variance to cover 2.4% of your domain.

---

I agree wholeheartedly with this post.

Most people seem to think along the lines of the OP, that +/- 2 is reasonable over 100 tries. This is understandable. You learn about percentages in math class, but you generally don't get a feeling about what the variance is really like.

After looking at tables upon tables of randomly generated numbers and percentages at my job, I know it's much worse than that. I agree that +/- 12 over 100 tries is way more realistic. As you say, 100 is not a large sample size at all. Ten thousand is a much better number.

Quote:

---

Originally Posted by Ed_N_Ants

You are aware that computers are not capable of coming up with random numbers... Nothing a computer does is random and there is some kind of method to come up with the "randomness"... There are different ways to convey this such as time, paths taken or algorithms but never are they random, they only appear to be...

---

Yes, though in the year 2013 freely available psuedo-RNG are more than accurate enough that even their inaccuracy should disappear over sufficient rolls. The danger of a "bad" RNG algorithm is usually either predictability or lack/abundance of runs, the first of which is only evident when you're trying to predict the next number in sequence and the last disappears over large numbers.

Where the errors are these days is usually in the seeding of the RNG, this is a logical error in the programming and usually represents itself as pulling identical numbers/sequences in a row. I have no idea how they seed RNG in an MMO, I'm sure there are multiple strategies out there. My initial thought is that you would attach the RNG object either the character/npc or off of the zone the character/npc is in at the start of login/spawn/server boot and seed it with some hash of the current time. Who knows though.

Edit:
Maybe current time + some ID number on the char/npc/zone hashed? Now I'm interested in their seeding strategy for multi user interfaces, damn you science!

Quote:

---

Originally Posted by roohan

I'm always not sure if it is just me or if it is really like this in the game. I have the feeling that touches on crafts fail more often when the condition is good compared to normal.

---

I've noticed that as well. for the first 18 levels on your launch craft you are stuck with a 70% touch and I hate seeing good in that phase of crafting because 90% of the time it meant the touch would fail.

As for general percentage, if you look in the gathering Achievements you will see 30, 300, 1,000, 2000, 4000 based on tier.

I am speculating but I think our crafting algorithm is tied to that. If you are doing a 41-50 craft and you have a 20% base HQ on NQ material, you can expect to craft 4000 of that item before you see the 20% pan out. I also think they shoved all the failures in to the the front of the pack because there is no chance this side of heavens gate anyone will craft 4000 items in a session. So the first 800 are subject to a much higher failure than 20% because the Algorithm would balance out after that and give you hundreds of HQ items in a row. Of course it makes the Algorithm useless as nobody will craft that much at a time.

tldr: We are being rick rolled.