% for melding materia is broken

[ispano]

Keyword "expect".

I was not saying the actual outcome would be exactly 41 every time.

And so you should not "expect" it either.

[Tiggy]

And so you should not "expect" it either.

It's only going to be on average 41 out of a 100 as well. Small sample sizes leave plenty of room for outliers of streaks to complicate the picture.

Which brings me to another point no one has mentioned. Not a single person here has properly recorded their successes and failures at each % of success.

For example. at 99% success I've done 200 synths. My data shows I have a 95% success rate from my 200 synths so far. Even this is too small of a sample size to be statistically significant.

This is the only kind of data that matters. Not anecdotal evidence for a couple of synths. That means literally nothing to statistics.

Proving the RNG is broken is a statistical problem. You must, through statistics, prove there is an imbalance. I won't believe a single complaint until people start recording this data to show if a true bais exists or not. Otherwise it's all subjective and anecdotal.

[Astralos]

For example. at 99% success I've done 200 synths. My data shows I have a 95% success rate from my 200 synths so far. Even this is too small of a sample size to be statistically significant.

Well, not exactly. Most social sciences hold that 30 is the generally-accepted minimum number for a sample size, but also that as you increase sample size beyond that point, statistical significance increases in tandem. 200 is a perfectly valid sample size from which to draw conclusions.

[Tiggy]

Well, not exactly. Most social sciences hold that 30 is the generally-accepted minimum number for a sample size, but also that as you increase sample size beyond that point, statistical significance increases in tandem. 200 is a perfectly valid sample size from which to draw conclusions.

However, this isn't a social science. This is statistics pure and simple. http://en.wikipedia.org/wiki/Sampling_(statistics) As this wiki article's sheer length illustrates it's not a simple as you seem to make it, and 30 is far from a realistic answer in statistics.

All that needs to be shown is that given successive samples the % success rate converges on the listed rate or fails to do so, and to show this data with a significant sample size for statistical significance. This is something not a single person here has ever even attempted. Anecdotes mean nothing. If you want square to look at their algorithm more closely then prove it's broken. Prove it with cold hard numbers.

Also, many people seem to forget how easy it is for square to prove it works. They have the generator. It's entirely likely, and probable, that the algorithm is surrounded by a unit test. It generates potentially thousands of random numbers within a range in just a second. Then a very simple statistical analysis is done to show the numbers generated are evenly distributed along the range within tolerances. It's so incredibly simple to prove that it's not surprise they have confidence their algorithm works correctly.

[Astralos]

However, this isn't a social science. This is statistics pure and simple. http://en.wikipedia.org/wiki/Sampling_(statistics) As this wiki article's sheer length illustrates it's not a simple as you seem to make it, and 30 is far from a realistic answer in statistics.

It doesn't matter - all empirical sciences use the same statistical models and guidelines, which is part of the reason why mathematics are considered a "universal language." I'm not sure what a wikipedia article's length has to do with anything (except that it's a form of the Argument by Verbosity fallacy).

30 is fine. 200 is better. 10000 is awesome.

[Tiggy]

I'm not sure what a wikipedia article's length has to do with anything (except that it's a form of the Argument by Verbosity fallacy)..

It proves that it's not as simply as just saying "30" as you seem want to do. It's clearly a topic in which there is much discussion, many opinions, and several methods to arrive at a useful number and just stating one number to rule them all is not an adequate answer to the sample size question. It wasn't an argument by verbosity fallacy at all since you missed the intended meaning. Just because I don't feel like breaking down each and every bullet point on that page doesn't mean what it contains was irrelevant.

When dealing with percentages in a range this wide 30 will not be significant enough. It will have nearly zero tolerance for streaks of any kind. That's not a statistic worthy of any significance.

[O-Deka-K]

It doesn't matter - all empirical sciences use the same statistical models and guidelines, which is part of the reason why mathematics are considered a "universal language." I'm not sure what a wikipedia article's length has to do with anything (except that it's a form of the Argument by Verbosity fallacy).

It does matter. Social sciences are about human behavior. Humans do not act in a completely random manner. They are affected by culture, environment, upbringing, etc. The threshold for what is considered an appropriate sample size can be lower. It depends on what you're measuring.

30 is fine. 200 is better. 10000 is awesome.

When talking about purely random numbers, a sample size of 30 is negligible. 200 is tiny. 1000 is still on the small side. 10000 is decent.

[Astralos]

It proves that it's not as simply as just saying "30" as you seem want to do. It's clearly a topic in which there is much discussion, many opinions, and several methods to arrive at a useful number and just stating one number to rule them all is not an adequate answer to the sample size question. It wasn't an argument by verbosity fallacy at all since you missed the intended meaning. Just because I don't feel like breaking down each and every bullet point on that page doesn't mean what it contains was irrelevant.

I'm sorry, I'm starting to get a little tired and frustrated trying to re-iterate the point that seems to be repeatedly flying over your head, so I'm just going to be blunt. You don't know what you're talking about.

For the last time: 30 is a perfectly valid starting point to determine statistical significance. But as that number increases, so does the certainty in which you can declare statistical significance - so whenever possible, try to use as large a sample size as possible.

Here are a few actual sources (i.e., not wiki articles) for you to learn more about sample sizes & statistical power:
1) Student (1908a), “The Probable Error of a Mean,” Biometrika, 6, 1–25.
(1908b), “Probable Error of a Correlation Coefficient,” Biometrika, 6, 302–310.

2) http://www.stat.ufl.edu/~aa/articles...n_binomial.pdf

Also, the vast majority of that wiki article you linked was TOTALLY IRRELEVANT to the discussion, because it talks about so many facets of probability and sampling. Thus, it was a proof by verbosity.

When dealing with percentages in a range this wide 30 will not be significant enough. It will have nearly zero tolerance for streaks of any kind. That's not a statistic worthy of any significance.

When talking about purely random numbers, a sample size of 30 is negligible. 200 is tiny. 1000 is still on the small side. 10000 is decent.

If you two have any actual evidence (preferably academic in nature) that 30 is now considered "insignificant" or "negligible", rather than just relying on proof by assertion, I'd be happy to take a look at it.

[Fawkes]

And so you should not "expect" it either.

Then you're saying the displayed % is completely and totally unreliable and pointless to display to the player as it means nothing whatsoever?

[ispano]

Then you're saying the displayed % is completely and totally unreliable and pointless to display to the player as it means nothing whatsoever?

No, i'm saying you don't actually understand how it works.

When something has a percentage chance of happening, you should always expect that the outcome will be random. You should expect that if you have, say a 75% chance to succeed, that you "should" get mostly successes, not that if it doesn't happen, something is wrong.