Time for a public test server?

[Shurrikhan]

Striking dummies, some modicum of group content capable of testing all basic functions (healing, movement, etc.), system changes, and then samples of whatever new mechanics or design philosophies they specifically want feedback on would be enough.

There's really no need for anything that could be called "spoilers", exactly, apart from skill additions—e.g. the same things we'd expect to have the same information on in text or screenshot form already.

[NanaWiloh]

If you want too know what kind of baggage that comes with PTR look at the WoW forums or other MMO's that use PTR's. Its mass crap storm of one group saying something is broken and one group saying it is not, leaving the developers to syphon through it all too figure out what is actually broken. I don't want see FF14 end up like other MMO's where the developers just stop listening cause they get tired of trying pull the real problems out of the pile false problems.

[Remedi]

Or the fiasco that was one balance patch in starcraft 2 last autumn, where they invited all pro players to test the new changes to the meta, the result was that said changes were ALL reverted 2 weeks after the changes hit live servers after 2 months of PTR.
I don't necessarily know who is in the wrong of that but it does tell me that PTRs do not garantee quality

[Alleo]

Next to a public test server for jobs and skills I really wish they have a place where they will show their ideas to some of the content before finalizing on it. I am not sure if Diadem or LoV would exist in that way if people would have been able to give their opinion about it before they had truly created it. I know its probably too much work and not that easy since you have lots of different views but a girl can dream, right?

(In one french interview they were asked about a vote after the end of a dungeon run. Like how they liked it and more, but Yoshida said that this would break immersion for the player so they will not do it, also they already have their data..really with all the things that exist ingame I am quite sure that there is enough to break immersion..heck you could just make this something that you do after your first run so people would not have to click it away every time.)

[Zumi]

SE had a test server for awhile on FFXI but not many used it at all. They actually had updates on it that didn't come out for like 2 years.

[Kazrah]

Or the fiasco that was one balance patch in starcraft 2 last autumn, where they invited all pro players to test the new changes to the meta, the result was that said changes were ALL reverted 2 weeks after the changes hit live servers after 2 months of PTR.
I don't necessarily know who is in the wrong of that but it does tell me that PTRs do not garantee quality

Usually player surveys asking what kind of content they like doing can fix that problem. PTRs may not guarantee quality, but there are measures that can be taken on both sides that can increase the odds of it being there.

[Shurrikhan]

If you want too know what kind of baggage that comes with PTR look at the WoW forums or other MMO's that use PTR's. Its mass crap storm of one group saying something is broken and one group saying it is not, leaving the developers to syphon through it all too figure out what is actually broken. I don't want see FF14 end up like other MMO's where the developers just stop listening cause they get tired of trying pull the real problems out of the pile false problems.

That's because they release it as a playground available to both subscribed and non-subscribed players for free gear; the climate therefore is even less useful to actual information from its broad strokes than the cacophony of the live servers. Compare that to closed beta testing, where Blizzard specifically asks leading guilds who know what they're doing, what apparent flaws will actually penalize choices in loadout or composition, by what margins, what feels "off" to a skilled player, and maybe even suggest what can be done about those issues, to join and test their stuff.

You can read Jeff Kaplan's comments on what would be necessary to make the PTR a useful testing ground in Overwatch. It's not an easy process. In the end any game's mechanics must be developed with its optimal outputs in mind, but a random testing bay is not going to spontaneously produce the player conditions necessary to find or make use of those.

But that doesn't remove the value of allowing those who can actually make something of the information given to do so. It just means that there's very little usable information discrepancy between theory-crafting from the paper data and from the practical data, so long as the paper data is willing to include fight conditions. In all other cases, the theorycrafting, done well, is usually more accurate to the outcomes of a given tier than the experiential findings of a PTR by people who, frankly, are highly divorced from optimal play. Their opinions on the gameplay, numbers taken out of the equation, is essential and likely even matter more than the voices of those who tend to theorycraft, simply because they, unlike dedicated performers, are the majority. But when people start arguing performance in based on experiential findings (or worse yet, without any objective data collection, e.g. by detailed parsers) in non-optimal settings, that's where **** hits the fan.

tl;dr: Your average engaged (e.g. willing to learn) players best informs on gameplay appeal to the overall playerbase. Your high performers best inform on performance. Mixing the two data sets unknowingly can greatly diminish test accuracy. This nonetheless does not invalidate that data; it just makes it harder to sort when the testing grounds are not well set or defined.

[Hieral]

They already do this. It's referred to as "early access."

[Lambdafish]

Its mass crap storm of one group saying something is broken and one group saying it is not, leaving the developers to syphon through it all too figure out what is actually broken.

XIV is already like this, except it happens on release, and is patched months later.

[Kaedan]

PTSs are generally not worth the time and money put into them.

The vast majority of people who play on PTSs never contribute to bugs or development in any way. They simply want to play.

As someone who has participated in many various tests over the years, and has worked in game development, I can safely say that the number actual, useful, contributing members of a PTS are less than 10% of the population of that PTS.