Real Math and Sq Enix Math - the twain shall never meet

I'm essentially a full time gatherer/crafter. I've noticed a significant disparity in the stated "success" percentages and real outcomes. Not enough room here to post in detail (I've kept a log for both crafting and gathering and have statistically significant numbers), but someone at SE needs to take a course in Binomial Probability. After farming 990 Rosewood logs today, that 93% success rate was suspiciously close to 78% -- and the likelihood of three successive failures (at 93%) is astronomical; yet, it happened seven times. Still love the game heaps, but how about revamping the crafting/gathering interface with accurate percentages!

Welcome to RNG

You know that random is random right? Even if it's statistically less likely to happen, it CAN happen.

Yup, I get it. I'm not really biotching about it -- I love the crafting/gathering in this game too much to get terribly upset, I just find it remarkable, random or not, that a 93% chance at success can fail sequentially as frequently as it does. It hints that this so-called randomness is weighted markedly against the player. On the flipside, when gathering, I'm so terribly pleased that 15% HQ chance is off in favor of the player! Too bad the same cannot be said on the crafting side.

Cognitive Bias

That 93% chance to succeed still has a 7% chance to fail every time.

Welcome to the world of RNGesus.

i wish you would post your data. would be interesting to see.

Quote:

---

Originally Posted by fortuona

i wish you would post your data. would be interesting to see.

---

Steel Ingots, using a macro with five Touch skills -- going for HQ ingots (was not using Standard Touch at the time).

100 ingots made -- 500 touches

Result: 181 failed touches or 36.2% failure rate

Two sequential failed touches occurred 41 times

Three sequential failed touches occurred 19 times

Five of Five touches failed once

Total times HQ failed when >98% : 8 times

Total times HQ succeeded when <20% : 2 times


Similar results occurred on my Weaver when making Linen Yarn, as did my Leathermaker when converting Toadskin to Leather.

Regrettably, I did not keep records when my crafters were at lower levels; however, the precipitous number of failed touches is what caused me to begin keeping a log.

Quote:

---

Originally Posted by Ashelia_Ferron

You know that random is random right? Even if it's statistically less likely to happen, it CAN happen.

---

That's the thing tho. This isn't true random. It's very hard to implement true randomness using computer algorithms. For all we know their RNG seed/algorythm could be completely bonkers, and the only way to verify it is with massive, and exhaustive tests.

In fact, I remember a dev once talking about their RNG seed having issues with repeats: It tended to repeat itself a bit too much. They said they patched it up a bit, but from the looks of it it's still repeating itself a lot. This is why it's rather common to see strings of 5x fails when statistically that should only happen once in a blue moon.

To the people saying "well random is random. It's not 100%" everyone gets that, the issue is the proc rates. When you think about and see how high the proc rate is for a 3%-20% chance to fail/NQ it's absurd. Since 3.0 the proc rates have worsened, and most of the people who are strictly craft/gather have noticed it. The real question is why does such a low chance to fail have a high chance to proc constantly, but yet the low % chance to succeed has an abysmal realistic proc rate? For example why can the 20% fail rate for SH2 + HT fail back to back 4/8 times in 1 synth for 3 separate synths but a 20% chance to HQ will proc maybe 1 out of every 200 or so synths? The same with gathering, 98% chance to gather and you gather 500 items but you miss about 45 attempts, some of them twice in a row. That 2% proc rate is pretty high. But yet your 12% chances to get an HQ only proc'd about 20 times during that whole session.
The game is too heavily reliant upon it's RNG system to keep people from progressing too fast. It's been stated by Yoshi P himself in interviews when they design something, for instance relics in 2.0, they have a time frame when they feel people should obtain/complete them. So really progression is gated behind RNG. On top of that we have the materia system where we need to meet certain requirements just to gather/craft things. That's all the materia is good for in all honesty, because stats and skills have no hold over RNG. Then there's the RNG of melding which is horrid. All in all you can have the most pimped out fully melded HQ gear and still fail as miserably as if you were in the NQ gear with a few melds. So is RNG a bad thing? No, it's not, not saying it is. But when it becomes something that is heavily relied upon and becomes a gate towards progression more so than skill, then it becomes a bad thing.

Quote:

---

Originally Posted by Kyros

This is why it's rather common to see strings of 5x fails when statistically that should only happen once in a blue moon.

---

You are assuming the 5x fails in a roll will be "spaced out" ... that's not how randomness works.

Well here's the dev response: http://forum.square-enix.com/ffxiv/t...Random-Comment

Do with it what you will

http://www.kodyaz.com/images/pics/ra...bert-comic.jpg

I'm no math expert here, but, isn't there an essential difference between a 93% chance of success, and a 93% rate of success?

There was a dev post on this awhile back, its surely buried in the 29000 threads in general discussion now, but its there.

Quote:

---

Originally Posted by Bishop81

You are assuming the 5x fails in a roll will be "spaced out" ... that's not how randomness works.

---

No actually, that is exactly how it should work, averaged over time. You can calculate how often does X scenario should present itself fairly easily. If it isn't behaving as expected, then your seed has a bias and is not truly random.

Also, RNG seeds can have quirks to them, because again: not truly random. Take for example this number sets: 25 50 75, and 50 50 50 repeat over and over. Both average to 50 and you can say that both are truly random, but one of them is not really functioning as a RNG for our purposes.

Quote:

---

Originally Posted by Kyros

No actually, that is exactly how it should work, averaged over time. You can calculate how often does X scenario should present itself fairly easily. If it isn't behaving as expected, then your seed has a bias and is not truly random.

Also, RNG seeds can have quirks to them, because again: not truly random. Take for example this number sets: 25 50 75, and 50 50 50 repeat over and over. Both average to 50 and both and you can say that both are truly random, but one of them is not really functioning as a RNG for our purposes.

---

Random by definition means "no pattern". Having the 5x fails in a roll spaced out as a characteristic would be a pattern.

Quote:

---

Originally Posted by Seig345

I'm no math expert here, but, isn't there an essential difference between a 93% chance of success, and a 93% rate of success?

---

Rate is just chance averaged over many trials.

Quote:

---

Originally Posted by Valkyrie_Lenneth

There was a dev post on this awhile back, its surely buried in the 29000 threads in general discussion now, but its there.

---

I posted it :P

It's 3 posts above yours

I have been crafting and have gotten NQ so many times while having 98% and 99% chance of HQ. It is just very frustrating with the RNG. R quad it just doesn't make sense in the Lore. A master crafter would not fail that much at making HQ when they have been doing it forever and are a "Master" in it.

Quote:

---

Originally Posted by Lyrinn

Rate is just chance averaged over many trials.

---

Yes, the point where I'm confused is, the system's 93% is referring to individual chance, not a rate of success, yet the OP refers to it as a rate, and then accuses the rate of being suspiciously closer to 78% than 93%. Seems like a mismatch of terminology here.

As infuriating as it is to fail on a 93% chance of success, I always took it to mean that an RNG was picking a number from 1-100, and that I'd succeed if it picks any number from 1-93, but 94-100 = failure. Random unfortunately means that even if the odds are in your favor, it doesn't mean you're guaranteed more successes than failures (although even in the OP's example, a 78% rate of success is still more successes than failures). As far as homework grades go, 93% is great, but if you go into RNG with that thought process, you're bound to be frustrated.

SE can say its 95%...all they want but we really dont know what thay have coded ..just because it says it doesnt mean that what they are really offering, dont put it past them to be sneaky...lol

tbh i think its wayyyyyyy off.

Quote:

---

Originally Posted by Mikedizzy

I have been crafting and have gotten NQ so many times while having 98% and 99% chance of HQ. It is just very frustrating with the RNG. R quad it just doesn't make sense in the Lore. A master crafter would not fail that much at making HQ when they have been doing it forever and are a "Master" in it.

---

This raises what I think is an important issue. Shouldn't there be a reward for experience? For instance, the more Iron Ingots a crafter makes the less likely their actions will fail and the more likely they are to create an HQ item. Persistence should be rewarded and would provide a richer gaming experience; more so than arbitrary RNG. In this way the mechanics can still control how quickly players progress. And as you say, this notion makes more sense in the context of lore/player immersion. LOL ... NOTHING is less immersive than that wretched sound when a crafting touch fails.

Quote:

---

Originally Posted by Seig345

Yes, the point where I'm confused is, the system's 93% is referring to individual chance, not a rate of success, yet the OP refers to it as a rate, and then accuses the rate of being suspiciously closer to 78% than 93%. Seems like a mismatch of terminology here

As infuriating as it is to fail on a 93% chance of success, I always took it to mean that an RNG was picking a number from 1-100, and that I'd succeed if it picks any number from 1-93, but 94-100 = failure. Random unfortunately means that even if the odds are in your favor, it doesn't mean you're guaranteed more successes than failures (although even in the OP's example, a 78% rate of success is still more successes than failures). As far as homework grades go, 93% is great, but if you go into RNG with that thought process, you're bound to be frustrated..

---

It's frustrating because of basic statistics - Law of Large Numbers says the rate will trend towards the expected value over a large sample size. When you do take a large sample and it's not remotely close to the expected value, it makes you wonder if something is up. Of course, this is where confidence intervals and stats tests come in to play but that's more work than anyone is willing to put in.

Quote:

---

Originally Posted by Bishop81

Random by definition means "no pattern". Having the 5x fails in a roll spaced out as a characteristic would be a pattern.

---

Exactly. Gosh. If you are finding patterns in your Random seed, then it really isn't random. They don't have to be spaced out, they can be wherever they damn want, as long as they fall within what's expected. That's why normality curves exists. If you are constantly having outliers outside your standard deviation, then your seed isn't working properly. it's as simple as that.

Here, a simple Wolfram example. Chance of 5 failures in a row is 1 in 3125. OP here had it happen in >500 touches. Likewise, chance of 3 fails in a row is 0.8%, or 1 in 125, and OP had it happen 19 times. I can't say much about the 5 failures since that requires a much bigger sample size, but the 3 fails in a row is such a huge outlier that really makes me question the seed working as intended.

On second thought, it seems he used no SHII, and I have no idea which touch skill he was using, so disregard that math.

Quote:

---

Originally Posted by Kyros

...

---

The OP's example is kind of vague.

Just exactly which touch is she using? Is Steady Hand I//II up?

I think OP is greatly underestimating his sample size in comparison to every crafting / gathering attempt that are happening every second. I'm quite positive that the rates even out somewhere across to board.

To gasp at an incredible 10 fails in a row at 99% rate of success, is to greatly underestimate the total number of things happening. Very improbable things happen... all the time.

Quote:

---

Originally Posted by Kitfox

To gasp at an incredible 10 fails in a row at 99% rate of success, is to greatly underestimate the total number of things happening. Very improbable things happen... all the time.

---

That's true. There are people who got their drops (whether ATMAs, Crystal Tower minions, Zodiac drops, raid loot, Triple Triad card) the first time they ran it. There are people who still haven't gotten something after going at it hundreds of times.

If you ask me, it makes for a very skewed system, where the lucky people sit in their corner and go "bah, the RNG isn't that bad, it only took me three attempts", and the unlucky people in many cases give up and decide it's not worth the stress. Personally, I much prefer RNG systems where you have a 'cap' on how unlucky you can be. Say you have a 5% chance of a particular drop, but if you haven't gotten it after 500 attempts, you're guaranteed to get it (numbers picked from the air). Or increasing chances for each failure (so those 5% might be 5.2% after the first attempt, 5.4% after the second, and so on).

Out of random curiosity, though... I wonder how large a sample size would be needed in order to 'even out' at the suggested probability rate.

Quote:

---

Originally Posted by Mallea

(..) For instance, the more Iron Ingots a crafter makes the less likely their actions will fail and the more likely they are to create an HQ item. (..)

---

In most games this is called EXP or just XP (Experience Points). While we have that in FFXIV too its just used as brickwall not for success rate. In older Final Fantasy games we can use Skills/magic and fill a bar, when reaching 100% we can use that same skill even when not being that job or when not having equipped that gear... There are even some games where you get stronger skills by using the older skills more often (think on FFVIII summons and friendship to them).

Quote:

---

Originally Posted by Mindylou

SE can say its 95%...all they want but we really dont know what thay have coded ..just because it says it doesnt mean that what they are really offering, dont put it past them to be sneaky...lol

tbh i think its wayyyyyyy off.

---

Lol yup. Never really know unless someone runs an extensive test. Just the other day I was mining adamantite for red scrip turn ins. It says 95% chance. I had three swings with my collectability over 470 and missed all 3. So frustrating. 95% and i miss 3 in a row.

Meanwhile, I've HQed multiple things at 1 or 2%. Random truly is random; sometimes it's in your favor, sometimes it's not.

Yeah I agree it doesn't feel like it corresponds to the % very well. Before I bad 650 GP I missed like 1 out of 6 of my collectables at least every single gather, no joke. At 95% its kinda like um ok. So now me and my friends just joke if it isn't 100% then its 50% chance no matter where your % is. Don't even get me started on hasty touch...

Quote:

---

Originally Posted by LunaHoshino

Meanwhile, I've HQed multiple things at 1 or 2%. Random truly is random; sometimes it's in your favor, sometimes it's not.

---

I take more issue with the times it happens in a row. How many times have you HQ your 2% item in a row? Likely never. Yet I see multiple fails of 2 and 3 in a row very often at 95%+. While yes it is that same chance every time, the odds to lose that many times in succession from a true rng situation is extremely unlikely.

Quote:

---

Originally Posted by Mallea

Still love the game heaps, but how about revamping the crafting/gathering interface with accurate percentages!

---

Someone correct me if I'm wrong, my stat skills are a bit rusty, but with a sample size that big and a confidence level of 99% that only leaves a margin of error of +/- 3.4% meaning that your data backs up the fact that the numbers we're seeing aren't close to the actual numbers. That, or you're some sort of extreme outlier. It leads me to believe that similar to crafting there's some sort of curve.

You know, even if SE's RNG is bugged (i.e. not truly as random as it should be), there is a very good chance that there is only one RNG that is shared among all the players per server.

In other words, the timing of your "attempt" also affects the numbers you get as other players are also drawing from the same RNG, adding yet another layer of randomness to the roll.

My worst luck by far has been 22 fails in a row on a 40% materia meld. I was actually super amazed after 10 fails and just kept getting more amazed with every fail......by 22 it was like wow.....just wow lol. I have also failed 50+ TIV STR materia melds in a row at 10% or 11%, I forget which exactly but I literally stopped counting after 50......might have been 60 or more!

Also at the other end of the spectrum I 5x melded a chivalric coat of fending with only 3 fails total with 3 T4's and 2 T3's........and right after I 5x melded a chivalric coat of maiming with 5x t4's with only 2 fails........5 fails total. I also hq'd the coat at 9%, luckiest coat ever, or so I thought. The very next day they added new gear with a patch making it go from best in slot to trash and it's been on the market for dirt cheap for like a month lol.

[QUOTE=Mallea;324786 I've noticed a significant disparity in the stated "success" percentages and real outcomes someone at SE needs to take a course in Binomial Probability. After farming 990 Rosewood logs today, that 93% success rate was suspiciously close to 78% -- and the likelihood of three successive failures (at 93%) is astronomical; yet, it happened seven times. Still love the game heaps, but how about revamping the crafting/gathering interface with accurate percentages![/QUOTE]


Oh yeah I aggree 100% oh wait that would be 70% SE match. I swear I think they still use an Abacus to figure these percentages. And I don't even have to keep track of stats to know there's something screwy with it. I can tell just by gathering every day...you guys always blame this on rng well SE's rng is messed up too then. When I go and gather an item that has 55% hq rate along with an extra attempt and I don't get one hq item and the next node is 15% and I get 4 in a row...yeah....