Making yet another post about this because of lazy devs

[Packetdancer]

then its time to make a ptr and let the players test things.

On this one point, we actually agree. There's huge value to PTRs -- or even to having a select group of players who have access to early builds and a single test world, if doing a full PTR that works for all regions is problematic. (Like the Evocati with Star Citizen, for instance, much as I dislike that system.)

I know some people scoff and are like "oh, that's just outsourcing the testing" and I suppose you can look at it that way, but:

• Software QA in general -- and game development QA in particular -- is not well-suited to really thorough testing, given how it's usually handled.
• Live environments and development environments vary greatly in online gaming; a development server is handling a much lower load, and is thus far less likely to see errors like I/O bottlenecks or database locking race conditions. (See also: housing placards not receiving lottery data.)
• Players will generally try weird things you never imagined anyone would try. (Is it rational for someone to try hitting a wall 50 times in Elden Ring to see if it broke? Probably not really. Did someone do it anyway? Yep.)

Making a test realm that's wiped frequently (but lets you copy your real character over after a wipe, to ensure people can test endgame content and whatnot) and letting players loose on it really, definitively will help a team find bugs triggered in ways they'd never consider in order to put into a QA plan.

Mind you, even a PTR won't catch 100% of bugs; for instance, a PTR probably would also have a low load compared to a real live world, so even testing the housing lottery there might not have had the situation arise that prevented placards from receiving data, and still let that bug slip through. But even if it's not 100% of bugs, a PTR will absolutely catch more bugs than purely internal testing will, and finding any non-zero number of additional bugs -- yes, there can be huge value in that.

[RitsukoSonoda]

On this one point, we actually agree. There's huge value to PTRs -- or even to having a select group of players who have access to early builds and a single test world, if doing a full PTR that works for all regions is problematic. (Like the Evocati with Star Citizen, for instance, much as I dislike that system.)

I know some people scoff and are like "oh, that's just outsourcing the testing" and I suppose you can look at it that way, but:

• Software QA in general -- and game development QA in particular -- is not well-suited to really thorough testing, given how it's usually handled.
• Live environments and development environments vary greatly in online gaming; a development server is handling a much lower load, and is thus far less likely to see errors like I/O bottlenecks or database locking race conditions. (See also: housing placards not receiving lottery data.)
• Players will generally try weird things you never imagined anyone would try. (Is it rational for someone to try hitting a wall 50 times in Elden Ring to see if it broke? Probably not really. Did someone do it anyway? Yep.)

Making a test realm that's wiped frequently (but lets you copy your real character over after a wipe, to ensure people can test endgame content and whatnot) and letting players loose on it really, definitively will help a team find bugs triggered in ways they'd never consider in order to put into a QA plan.

Mind you, even a PTR won't catch 100% of bugs; for instance, a PTR probably would also have a low load compared to a real live world, so even testing the housing lottery there might not have had the situation arise that prevented placards from receiving data, and still let that bug slip through. But even if it's not 100% of bugs, a PTR will absolutely catch more bugs than purely internal testing will, and finding any non-zero number of additional bugs -- yes, there can be huge value in that.

A bug like the one the OP is complaining about actually would have a high probability of being missed on a PTR as well since it's specific to an interaction with a specific skill on 1 class that is only used in PvP which in itself isn't very widely used by players of the game. The FC mate that told me about it also said it might be specific to the PS4 platform as they've been told by other BRD players on PS5 and PC that they haven't experienced it and they play the game on PS4.

[Mirhd]

... People have already made the (inaccurate) claim, time and again, that we are "testing" the game for the devs because they "can't be bothered"... when FFXIV has some of the best QA I've seen in an MMO, as serious bugs rarely make it to live. Yes, rarely. The forums' personal bias towards making mountains out of ant hills is not reflective of reality, and even as someone who has never worked in game development, I can still see the difference between good QA and poor QA. People would still complain even if there was a PTR, and you all arguing in favor of a PTR know it.

I also don't care for it on a personal level, as the PTR in WoW ruined any sort of surprise; even if you avoided it yourself, chances are you would see/hear about the new content elsewhere and have it spoiled as a result. Not to mention people demanding unreasonable perfection during runs because "it was on PTR! you should have done it/looked into it!"... Well, excuse me for not wanting to spoil myself, I suppose? I love the dumpster fire runs that happen every time during the first week of a new patch here... everything is new to everyone, and we're all on the same playing field. Hilarity is inevitable.

A bug like the one the OP is complaining about actually would have a high probability of being missed on a PTR as well since it's specific to an interaction with a specific skill on 1 class that is only used in PvP which in itself isn't very widely used by players of the game. The FC mate that told me about it also said it might be specific to the PS4 platform as they've been told by other BRD players on PS5 and PC that they haven't experienced it and they play the game on PS4.

Yeah. It's physically impossible to catch everything (even serious bugs can't always be caught!), and it's honestly amazing that anyone still tries to argue otherwise...

People complaining about the lack of love for Hrothgar (or proper instanced housing...) etc., is fair. The devs are aren't lazy though, good grief. Anytime I see someone slinging that word around, I immediately think said person is simply incapable of seeing anything beyond their own desires... such as the OP. The more and more she/he post, the more I think OP needs to do some serious self reflection, but this is unlikely to ever happen...

... Japanese companies, and software companies in general do not run on "lazy"...

Yeeeeah... there's a reason Japan is known for its work culture... because of how unhealthy it is. I've heard it's been getting better, but for crying out loud... "lazy" is NOT a word I would use to describe the devs here. Tone deaf at times perhaps, but not "lazy".

[KuroMaboroshi]

I also don't care for it on a personal level, as the PTR in WoW ruined any sort of surprise; even if you avoided it yourself, chances are you would see/hear about the new content elsewhere and have it spoiled as a result.

This is why, though I am on principle not opposed to a PTR, it would have to be for mechanical features only. Sure, let people test PvP abilities, potency changes, etc.. But anything content-driven should be kept well out of it. No MSQ, no new raids. Let players test systems, but not content.

Which sounds rather unlikely to happen cause the dev team already doesn't like people pre-judinging their balancing decisions (hence why they keep it out of patch notes until the last possible moment) and it would force them to maintain a completely separate PTR branch of the game that would likely require extra effort to maintain.

[Jeeqbit]

... People have already made the (inaccurate) claim, time and again, that we are "testing" the game for the devs because they "can't be bothered"... when FFXIV has some of the best QA I've seen in an MMO, as serious bugs rarely make it to live. Yes, rarely. The forums' personal bias towards making mountains out of ant hills is not reflective of reality, and even as someone who has never worked in game development, I can still see the difference between good QA and poor QA.

I had never seen a single bug for myself in years of playing before covid. I forget where I saw this but I remember there were over 100 people testing the game to find bugs.

https://forum.square-enix.com/ffxiv/...9&d=1613761434

https://forum.square-enix.com/ffxiv/...=1#post5507820

[Packetdancer]

It’s ok to post in other threads other people have created on topics you keep bringing up OP.

In fact, it's more likely to be productive. Moderators have said before that particularly active threads are likely to get noticed by the mod team, and if they're noticed by the mod team they're more likely to be taken to the developers.

Posting 87 different threads on the same complaint likely doesn't show up on the moderator "active threads" dashboard, and probably just becomes noise among the signal; one thread growing to 87 pages is almost certainly more likely to get noticed.

... People have already made the (inaccurate) claim, time and again, that we are "testing" the game for the devs because they "can't be bothered"... when FFXIV has some of the best QA I've seen in an MMO, as serious bugs rarely make it to live.

I will concur with this.

The fact that I think SQEX could demonstrably do better with their QA/testing should not be taken as a measurement of flaw against other game companies; I think all game companies -- heck, nearly all software companies -- could do far better on testing. Going into embedded systems has really made me a much more firm believer in better testing practices than when I was a game dev for my day job; the vast majority of people rarely update the firmware on various devices they have, so if you are making a medical device or a diving rebreather, you really want to make sure it works. Admittedly, the consequences there -- someone potentially dies -- are far greater than with a bug in a game, but that still doesn't mean there couldn't be some improvement in test practices used across the software (and game) industry.

That said, SQEX's record of software bugs is no worse than many game companies -- and better than quite a few, even without the complexity of the online ecosystem that an MMO entails. (And their track record on fixing major bugs quickly is way better than many.)

I also don't care for it on a personal level, as the PTR in WoW ruined any sort of surprise; even if you avoided it yourself, chances are you would see/hear about the new content elsewhere and have it spoiled as a result.

I also concur on this. While there's a lot of benefit to a PTR in terms of providing better (or at least, more diverse) test coverage, it absolutely ruins any surprise on storyline or plot which is so key to this game's intended experience. Sure, there are things in MSQ that could have been caught in a PTR; I'm thinking of a certain solo instance which concludes with a button-mashing Active Time Maneuver that -- unlike basically every previous one -- did not lock out legitimate keyboard input at launch, thus potentially causing you to open a bajillion windows in the game rather than actually giving input to the ATM. But I think the benefit of getting to discover the story relatively naturally rather than having it spoiled outweighs those handful of situations.

And it's a fair point to note that the difficulty of splitting out just new systems or mechanical changes without including content is a significant extra effort; it is one reason that a lot of companies don't do a PTR, if they don't want to spoil story and decide the time saved and test coverage gained is not worth the expense in time and effort to make a mechanics-only public test. Honestly, I suspect it is probably a factor in why SQEX doesn't, which... I mean, I can respect that, even if I do think a PTR for testing mechanics would be of huge benefit to any MMO, this one included.

[Mirhd]

I had never seen a single bug for myself in years of playing before covid. I forget where I saw this but I remember there were over 100 people testing the game to find bugs.

https://forum.square-enix.com/ffxiv/...9&d=1613761434

https://forum.square-enix.com/ffxiv/...=1#post5507820

The one time I encountered a bug since I started was... well, with the current "0 bidders/winner is 0" issue, as the plot I bid on for relocating was hit with it. People really do underestimate just how much work goes into a game (MMOs are even worse in that regard, obviously), and how many bugs do, in fact, get squashed before release (or open alpha/beta). Expecting no bugs to ever get through is... well, I have a lot of not-so-nice words for it, but I'll settle on "naive."

snip

As you and others have mentioned, I could see a PTR just for mechanics, fair enough... divorcing that from everything else though, and maintaining it... Honestly, I don't see the the gain outweighing the work expended, no. More testing is never a bad thing, but it gets to a point where there's only so much you can test in closed environments and you just have to release it... and possibly still end up with a potentially serious bug on live, even with a PTR... But, uh, I feel weird (as a laymen) saying that to someone who actually knows how this works, ha.

Anyway, it just gets my goat when people throw around words like "lazy" when that doesn't apply here. As I've said before, people can express discontent, but how you do it matters... Something I don't think certain posters OP will ever accept.

[Jeeqbit]

The one time I encountered a bug since I started was... well, with the current "0 bidders/winner is 0" issue, as the plot I bid on for relocating was hit with it.

And it turns out this wasn't entirely a bug but inefficient server processing that caused a backlog, so a lot of the results weren't ready to be communicated in time for the end of the lottery. Rather than fixing a bug the solution might be improving how efficient it is.