No Public Beta Testing: good bad for the game?

Quote:

---

Originally Posted by Espon

These are design issues, not bugs. The PTR is there to find bugs so that they don't make it into release, not for huge sweeping job changes. Those kinds of things are done in patches. Making such mass changes in the PTR would likely create more bugs and delay the release date. Eventually they'll have to release the patch and people will still complain because things aren't quite balanced.

---

PTR's exist not only to find bugs, but to test implementations of new features/systems.

Quote:

---

Originally Posted by VirusOnline

The issue with this is that it assumes that all feedback is good feedback.

---

Indeed, I'm of the belief that all feedback is good feedback, even if you can't or won't always use the feedback provided.
What I'm assuming, actually, is that there's someone, perhaps (but not necessarily) a QA Engineer, or a group of such people, who are capable enough to either parse through and utilize the feedback to further their goals or "mission" for the software in question, or who are capable of organizing and providing specific tests that must be undertaken by subjects in both a controlled and uncontrolled capacity.

To be honest, I believe that it would be better (and, frankly, prudent) for a neutral 3rd party group of professionals to be consulted instead. In-house testing, especially in large companies, has a tendency to just be a circle-jerk experience, especially in companies that have issues with standards surrounding seniority.
That doesn't mean there isn't a place for public testing, however; for example, public test realms would allow them to potentially determine certain issues like what we've had emergency maintenances for since the game released (and we've had a few), determine job balance and performance issues when they actually hit the playerbase (and isn't being tested in a controlled environment with company-approved or company-dictated behavior) prior to the official release, among other things, to help keep a smooth experience. Both the application and expectation of what can be tested should be kept in mind when proposing a public test, as you're asking for the average (and sometimes the below and above average) layman to provide a sample of data to work with.

But that's just my opinion.

Quote:

---

Originally Posted by Velnora

PTR's exist not only to find bugs, but to test implementations of new features/systems.

---

This right here.

As I highlighted before there has been more than one system change where it was announced and the playerbase immediately knew it was a bad idea and was vocal about it.

The FFXIV devs still released them in their bad state only to have to walk them back later. Earlier detection is ALWAYS better. More relevant data is ALWAYS better.

An ounce of prevention is better than a pound of cure.

Personally I don't like knowing everything beforehand. But utilising beta testing would allow developers to get direct feedback and data on their own classes and how well they perform, which in many circumstances could allow for players to present feedback on certain design flaws, or what they may deem as a flaw. It's essentially allowing for some fairly rudimentary testing at what would essentially be 0 employment cost.

For example, what they could do.. Instead of ruining the trials for example, and potentially story therein, they could give a very baseline testing ground, but to also upscale some older fights, both savage and extremes, which would allow them to get nice data, particularly when you factor in job synergies.

The only concern behind such changes though, and as is the case with any form of testing is.. Whether the feedback could then be fed back to both the designers and the developers in a due frame to allow them to make the necessary recalculations and adjustments based on feedback. In essence.. Having the ability to test something thoroughly, and then having the time to make all necessary adjustments, changes and recalculation is an entirely different slugfest.

Quote:

---

Originally Posted by Illmaeran

It's a good thing. There was far too much advance knowledge in WoW and in some circles expectations that you'd play on the PTR to practice for the actual release version.

I much prefer SE's *solid* in-house QC and no-spoiler launches.

---

Nothing more fun joining a raid on "release" only to be met with the expectation that you have watch detailed ptr videos of them.

Quote:

---

Originally Posted by Deceptus

This right here.

As I highlighted before there has been more than one system change where it was announced and the playerbase immediately knew it was a bad idea and was vocal about it.

The FFXIV devs still released them in their bad state only to have to walk them back later. Earlier detection is ALWAYS better. More relevant data is ALWAYS better.

An ounce of prevention is better than a pound of cure.

---

This is true. But the question is, is a PTR a good tool to detect problems earlier? In my experience, no it is not. Because in most cases players cannot be good testers. Maybe they will find randomly some "fall through the world at some position on the map"-like bugs or "tooltip says 30% increase but in real it is only 15%"-bugs. And the last category of bugs you can catch with unit tests. You can only be a good tester if you know how the results should look alike in the end. But players cannot know it when the content and the mechanics are new.

Cheers

I'm fine with how they do things in regards to this because if something major breaks they will fix it right away, which is more than I can say about SEGA and PWE heh.

Quote:

---

Originally Posted by GrizzlyTank

Nothing more fun joining a raid on "release" only to be met with the expectation that you have watch detailed ptr videos of them.

---

You can create your own group, and you can opt out of being part of the ptr.

With that being said though it really falls into the hands of the developers as to how useful this would be vs it being a waste of resources and time. It could be useful in other aspects as well such as server stress testing which may or may not have helped with this recent expansion launch.

If you take blizzard for example they have the PTR and Beta test but hardly use feedback provided from these tests. So it becomes rather a pointless implementation.

Maybe a public test server for just new jobs could be useful. Like, it has existing content only, but new jobs implemented, with the ability to just autocomplete class quests, if they exist.

But public testing for new content? No. Not remotely necessary and also would be a problem with spoilers. Definitely good we don't have it, imo.

Quote:

---

Originally Posted by IdowhatIwant

You can create your own group, and you can opt out of being part of the ptr.

---

Blind on release is the current baseline experience for players and it should remain such. You shouldn't have to create a special group to experience brand new content without other players' expectations weighing you down. That's the advantage of the current setup, everyone is on the same page at the start.

Naw, I'll pass. Don't need a bunch of players beta testing content to just leak all the "new" content on "release". Works better for FPS or other games with minimum story. Not all feedback is good feedback and can lead to broke and disjointed story/gameplay.

All the MMO I play with public test.
Always end up catering to those who cry the loudest.
It usually not the best outcome, just the most popular one. It broke balance between jobs/class, shift dev time to certain content
Eventually those who cry loudest take over the game

Quote:

---

Originally Posted by Awarewolf

I am curious how this community feels about the lack of a public beta test realm. On one hand, I see the advantages: it’s FUN for us all to figure out the story and jobs on our own. I had so much fun figuring out how things changed and hopefully improved for my favorite classes. It even got me to try Bard, which is my favorite job this expansion.

But, it also meant that so many jobs were released with clearly unfair advantages and disadvantages compared to other jobs, something that would come up on a PTR.

Do you feel FFXIV content would improve for having a PTR?

I think it would, but only if they let us test the new job/job changes on the previous expansion’s content. I don’t want story spoilers or to even see a new zone.

---

WOW has suffered from having a PTR for years. Not only does it blow the entire expansion a year in advance, but they ignore feedback on top of that. Plus you can't expect most of that feedback to even be good. There are people who can't figure out mob facing in a game that has like three different indicators that show you where it's pointed, and they're going to give feedback about mob facing that is a huge waste of time. Now expand that to every other facet of the game.

SE has professional QA, probably the best QA in the entire world right now. A PTR would only be seen as a negative, from every angle.

I'd rather not have to do the developer's job for them.

Quote:

---

Originally Posted by Misutoraru

All the MMO I play with public test.
Always end up catering to those who cry the loudest.
It usually not the best outcome, just the most popular one. It broke balance between jobs/class, shift dev time to certain content
Eventually those who cry loudest take over the game

---

ARR exists because of public outcry. I don't think you're really thinking this through enough. D-

Quote:

---

Originally Posted by Illmaeran

I much prefer SE's *solid* in-house QC and no-spoiler launches.

---

So solid that it frequently totally misses very obvious issues which are immediately picked out by the playerbase (like Anatman opener for two patches in a row, or removal of Energy Drain from SCH, or just entire lily system on 4.0 WHM). So solid that it screws up the expansion launch balance like clockwork and then fixes it over the patches, after they actually get to see the jobs tested by the playerbase.

While letting us play the entire expansion/patch the same way as WoW does it, would be lame due to spoilers - and rather pointless, as they usually do get content balancing right, except for a couple occasional exploits (o5s wagon cheese comes to mind, still kind of baffled why it even happened), thing is - there is no need whatsoever for full content testing like that.

The issues XIV usually runs into are with job design and balance and that can be tested entirely without any spoilers they don't give us normally. They already show us a zone and a dungeon (during media tour) as well as a normal mode trial (during Fanfests) and people get to play those. They did skip the trial with EW, because there were no "real" Fanfests due to covid and I guess it kinda worked out, as both were rather spoilery this time around, but it was an anomaly.

Just set up a PTR that works like the media tour build, except with an EX trial added (same one that was already shown as normal during Fenfests, so that it's not any additional spoiler). That is pretty much all you need to test what you need and it wouldn't show us any spoilers we wouldn't have seen otherwise.
For patch changes can of course just let us test them on content that's already available on the live servers. Easy.

It isnt good all people do is complain about things not being polished and dont get the idea of a beta

Quote:

---

Considering what would happen in the wake of WoW PTRs, I'm surprised you even have suggested adding a PTR here.

WoW players: classes are badly balanced
WoW devs: blame the PTR testers for insufficient feedback

WoW players: this content is bugged
WoW devs: blame the PTR testers for insufficient feedback.

WoW players: look- here's all this tester feedback that had been posted in the PTR forums about these very things
WoW devs: [delete the PTR forum posts]
WoW devs; We received no such feedback

At least the FFXIV team has to hold itself accountable if there are problems instead of blaming unpaid volunteers given no guidance or oversight.

---

Can you show me Q&A where devs said that players fault ? You kind of sounds like one of this players who never played the game but go no problem to s...t on it.

No, we don't need a ptr here. Thankfully, the devs feel the same way.

Quote:

---

Originally Posted by Alleluia

Blind on release is the current baseline experience for players and it should remain such. You shouldn't have to create a special group to experience brand new content without other players' expectations weighing you down. That's the advantage of the current setup, everyone is on the same page at the start.

---

This only applies to those competing for world first in actuality. And the competition for world first consist of stream sniping strats of who ever cleared the content first. With in the first week pug groups are expecting players to have seen a video or guide. This is far from a baseline experience for most players.

We don't get paid to do QA so why would Square Enix employ us to be testers? I think Square Enix is actually really nice for not making a public test server like other games. They treat us more like family rather than slaves with money. Plus, it avoids spoilers and people raging over tentative things. It would lengthen the development cycle and likely cause the community to be always angry. I've seen it happen before in various MMOs. SE is amazing with transparency and their QA work is pretty top notch. They were a bit delayed with 6.0 (only by like half a year lol) but we understand why and as a result they missed quite a few bugs and that's fine. No game is perfect and no one should expect it to be. Classes are in a perpetual state of balance and they try their best to make it as balance as possible over time unlike some other games I wish to name but won't.

I'll be happy to report bugs when I play because it's the right thing to do. I reported the fending choker glitch where the color was wrong 2 weeks ago and they fixed it by the previous patch. The devs listen!

I would say that a public test should only be for job changes and thats it. Nothing else in the data to datamine etc

The state of tanks and healers for a long time is why

Really dont think SE needs us for a beta testing of content. And pretty much everything else has been said what i wanted to say with the data mining and SE doing a very good job without us in terms of beta testing content.

Not sure how seeing all the content you’re gonna get and testing it before it’s released for the devs can ever be a good thing. Spoil your own surprise and do their jobs while not being paid to do so all the while actually paying for the pleasure to do this? No thanks

Quote:

---

Originally Posted by Recon1o6

The state of tanks and healers for a long time is why

---

No testing would do anything about that because it is that way by design for years now.

Quote:

---

Originally Posted by Retrisar

Can you show me Q&A where devs said that players fault ? You kind of sounds like one of this players who never played the game but go no problem to s...t on it.

---

You sound like someone who doesn't have a clue what was going on with PTR because you weren't participating unless maybe as a "ooo I get early access" participant that would loudly complain then never log back in the moment they hit the first game breaking bug.

Can't link what Blizzard has long since deleted under the guise of "forum improvements". All those blue posts get wiped right alongside all the player feedback they ignored.

As for not even playing the game, I played WoW 2007-2018. I beta tested in MoP and WoD along with doing testing on several patch PTRs. I wasn't one of the players who goes in with the "gee, I get to see things first!". I was testing stuff, trying to break stuff, repeating stuff that did break on multiple characters so I could get an idea of what conditions were in common before I submitted detailed bug reports, trying to find work arounds for the game breaking stuff so other testers wouldn't be stuck trying to get to content gated behind what was broken.

Spending hundred of hours doing all that just to see many of those problems still in game at launch and the Blue posts going "there wasn't any feedback" was frustrating in the extreme.

If I'm crapping on the developers, it's because they were crapping on us testers who were trying to help create a good release experience for our fellow players.

Quote:

---

Originally Posted by Recon1o6

I would say that a public test should only be for job changes and thats it. Nothing else in the data to datamine etc

The state of tanks and healers for a long time is why

---

The devs have 4+ years of feedback on the state of tanks and healers and things have only gotten progressively worse.

I like how it is now. I think the Game is extremly bugfree even without Beta Servers. And so its a suprise for everyone on Patch day

So the complaint everyone is making is if there is a PTR then I will be spoiled.

But, you don't have to look up anything on the PTR, play the PTR, or look up any leaks. If you do so then you have chosen to spoil the content for yourself.

Again the PTR is a tool more for the devs than the players anyways. If they devs need it and can utilize it good, if they don't then they likely won't implement one.

Quote:

---

Originally Posted by Misutoraru

All the MMO I play with public test.
Always end up catering to those who cry the loudest.
It usually not the best outcome, just the most popular one. It broke balance between jobs/class, shift dev time to certain content
Eventually those who cry loudest take over the game

---

MMOs cater to the loudest, as they then to be the larger group. This game has more casual content and heavily caters to casual players as they are the loudest and largest group. Which is why harder content and class design has suffered a bit over the past two expansions in this game. But the point is none of that had to do with a PTR.

AS a former wow player, definitely good for the game
Makes it so patch isnt solved before it even comes out, lets raids be exciting on launch, and it instills confidence in the devs that the game despite not having a ptr is forever less buggy than wow ever was, with one.
Just my 2 cents

Quote:

---

Originally Posted by IdowhatIwant

So the complaint everyone is making is if there is a PTR then I will be spoiled.

But, you don't have to look up anything on the PTR, play the PTR, or look up any leaks. If you do so then you have chosen to spoil the content for yourself.

Again the PTR is a tool more for the devs than the players anyways. If they devs need it and can utilize it good, if they don't then they likely won't implement one.



MMOs cater to the loudest, as they then to be the larger group. This game has more casual content and heavily caters to casual players as they are the loudest and largest group. Which is why harder content and class design has suffered a bit over the past two expansions in this game. But the point is none of that had to do with a PTR.

---

Well thats definitly not true. Its mostly a loud minority. Most Casuals don`t even post at all and just silently enjoy their game. The are definitly not the loud crowd.

A PTS doesn't really work for narrative driven games.

Personally i think SE does a great job doing that stuff in house. I dont think i would want to see that change and honestly i know everyone has opinions on what they think is best for a class so how much good would really come from a PTR?

Quote:

---

Originally Posted by Floortank

WOW has suffered from having a PTR for years. Not only does it blow the entire expansion a year in advance, but they ignore feedback on top of that.

---

This second point is actually relates to a really good one I didn't think of much before.

The PTR sets an expectation, and then in development when they find they need to something different for either good or bad reasons - either way players feel shorted because the outcome differs from the expectation.

SE's QA, as you note, is really good.

They don't always get it perfect on the x.0 patch, but a few patches in everything is usually smooth sailing. Not every class runs the expansion ideal - but we never have truly F- tier or SSS+ tier classes like WoW despite it's PTR seems to always suffer from.

People here complain about issues with classes like DRK and SMN that WoW players would only wish they could have - because by WoW standards (where half the classes are simply not invited to hard content period) those 2 are among the better balanced...

And despite the lack of a PTR, the SE team somehow manages to give us content we like on a regular basis. They don't have to go on stage and tell us "what you asked for is not what you actually wanted, you will want this, and ya'll can play it on your phones too..." ;)
- They just seem to be quietly involved, as well as all the fanfests where they listen more than they talk... and so a PTR has never been am issue here.

Add me in to the chorus of "no thanks."

Early Release was essentially the public beta for Endwalker, as it has been for the expansions before it. There are just some bugs that cannot be caught no matter how good your automated testing and QA team, such as Raubhan Ex (which was an unfortunate combination of higher than expected traffic + a lack of story branching shoving everyone into the same zone at the start. They fixed this in both ShB and EW to spread people out better before the first dungeon, as opposed to starting out with a solo duty 5 quests in that had every single player having to suck up an instance server..... oops.)

Other than those kinds of things, their in-house QA team does a tremendous job. I do a little QA as part of my job as a business analyst, and it is sometimes frustrating to spend 4 hours trying to recreate an elusive bug you saw ONCE and then never seems to happen again. (This morning I finally figured out it was happening because of an auto-fill on the end user's browser. Oh my god, you work in a hospital turn off your flippin auto fill!)

Unlike many western devs, Creative Business Unit III actually has good Q&A.

This isn't required for this game.

Nope, I think they do a good job on their internal QA. And there's a danger things would get spoiled. Don't really think this game needs it.

Quote:

---

Originally Posted by Awarewolf

I am curious how this community feels about the lack of a public beta test realm.

---

The game rarely has bugs on release and everyone is on equal footing on launch. If it ain't broke don't fix it. I enjoy it as it is now.

The only thing that would really need to be tested before a launch are the servers, ahah ! xD
FFXIV dev team do a serious good job at keeping the game bug free on launch, and even if they did have bug they fix it days after it was reported so we don't really need any beta test

I could see it working as long as it was completely walled off. You copy over your character (or don't even copy over at all but just get given a blank slate like in the media tours), get loaded into the Gridania Inn room, and have access to nothing but DF with a few instances. Maybe one solo with just a training dummy, maybe the most recent Unreal fight, and maybe one of the most recent EX or Savage fights. That's it.

Of course, even if the testing portion was doable, the development process would have to be adjusted to parse and incorporate the feedback on some level. We already know they basically have things scheduled like clockwork so this may actually be the harder part.

Quote:

---

Originally Posted by Awarewolf

Do you feel FFXIV content would improve for having a PTR?

---

PTR was was awful. It gave Blizzard the excuse to lay off their QA team and to use their players as unpaid beta testers. Actually, it was worse: you're paying $15 a month to beta test a buggy game, since WoW patches haven't released bug free in years. Blizzard also used the PTR as an excuse to blame their customers for a buggy, unfun mess. Blizzard also does not heed player feedback. People told Blizzard that the RNGendaries in Legion were terrible, and Blizzard shipped it anyway. People told Blizzard that the Azerite power was terrible, and Blizzard shipped it anyway. People told Blizzard that Chorghast was terrible, and Blizzard shipped it anyway.

The other problem with PTR is that the story is spoiled for you 4-5 months before the patch finally comes out.