The Disconnect between Players and Developers: Bug Reports Not Taken Seriously?

[Kitfox]

Hello,

Lynn Nuvestrahl, the Machinist theorycrafter, here again. I'd like to talk about improving the bug report workflow, especially for bugs regarding game balance.

TL;DR: Bug Reports for game balance issues require excessive documentation to be properly investigated and the investigations that do happen are sloppy.

I recently made a couple of bug reports about such issues that were marked as "Unable to Verify" along with today's patch:
-Tornado Kick: http://forum.square-enix.com/ffxiv/t...in-the-tooltip
-Upheaval: http://forum.square-enix.com/ffxiv/t...e-is-incorrect

I find this disturbing because I went to great lengths in both of my reports to very clearly indicate how to reproduce and observe the effects of the bugs. For those unfamiliar with the "Unable To Verify" category, this is the description:

Bugs that have been reported but cannot be replicated even though they have been investigated and tested will be moved here.

These are bugs that have been confirmed independently by multiple players. Furthermore, the Upheaval bug is not even the first time it has been reported, the first time being almost a year ago (details: http://forum.square-enix.com/ffxiv/t...en-in-Defiance). When the bugs can be confirmed even by untrained people simply by observing the game, it shouldn't be that hard for devs to catch them.

Especially in the case of the Tornado Kick bug, with current rotations MNKs can gain hundreds of potency per minute by properly utilizing the skill. If it has the wrong potency, it's a bug that actually breaks the job balance, as most buffs and nerfs are within the 1-2% dps adjustment range and this is most certainly well within that. Upheaval is a less severe example since most of the time tank stances are not heavily utilized in endgame, but nonetheless, the difference of hundreds of potency should be easily observable.

It's important for game balance bugs like these to be tested carefully because there's only a handful of people who even notice these things, and without proper investigation they never get fixed. A good example of this is the MCH Flamethrower bug. The skill was bugged since Stormblood launch, presumable for about 10 months. It was fixed but only after it was reported multiple times through different channels. Here's a couple of them:
-http://forum.square-enix.com/ffxiv/t...or-Direct-Hits
-http://forum.square-enix.com/ffxiv/t...-not-Crit-Dhit
-http://forum.square-enix.com/ffxiv/t...tisch-Treffen.
The bug report was not accepted until this particular report that stacked up excessive proof about its existence:
-http://forum.square-enix.com/ffxiv/t...-normal-damage

Now, I can think of a few explanations for the problems:

1. SE is too busy to deal with the bugs because of the expansion and they were pushed aside. Fair enough. Why weren't they moved to "Confirmed" or kept in "Accepted" category until solved?
2. The Bug Report team is ill-equipped to deal with these sort of bugs or there's a disconnection in the report pipeline that doesn't allow these bugs to be properly checked. Can the workflow be improved to better account for this bug type?
3. The operational team working on bug fixes actually doesn't know how their own code works or is supposed to work.

I truly hope it isn't the 3rd option, but it wouldn't be unheard of as we've seen a couple of times players coming up with more efficient skill rotations than the dev team has expected, here for example:

In my previous answer I mentioned that we check DPS against unmoving targets and balance jobs while considering the ideal action rotation. However, everyone has put in great effort studying ninja and has come up with a rotation that is slightly more effective than what we had imagined allowing for a slightly higher percentage of DPS.

Please Square-Enix, it takes hours to document bugs properly like this and seeing them go to the pile of unsolved issues without proper investigation is such a slap in the face. Is there a problem in the bug report processes to deal with these sort of balance bugs? What can we do to make it easier for you address these issues? Do we really need to mobilize dozens of people every time to be taken seriously?

I'm honestly running out of ideas at this point when spreadsheets and graphs no longer work for you. Especially in the light of the recent Ungarmax scandal I find this very depressing.

[Gralna]

Many bugs that make it past launch that aren't "critical"usually get placed into a bucket that's basically labelled "fix if you have time". It's very easy for these issues to get completely forgotten about by both the dev and qa teams, especially if the issues are not kept up to date regularly on their internal systems or they might get backburnered unless there is a surge in CS reports about it (i.e. ungarmax exploit).

Or it could just be that the issue is irrelevant due to future changes that are coming, at fanfest Yoshi mentioned tuning each job again for 5.0. Might be that issues with certain skills are being slated to be fixed, modified, or removed during that time. Why spend dev time fixing a minor issue, that in 6 months won't even be relevant?

As far as the category change, I'm not sure. That'd be something solely up to the devs to reveal, but don't think I've ever seen a dev officially post on the forums aside from messages from Yoshi and Koji.

Could also be that the issue has already been fixed on the version the devs/qa is on, so they weren't able to replicate it on their end.

[Raldo]

Since your concerns mainly focus on tooltips being incorrect, you could try posting in the localization forums. It's definitely not the right place for it, but bark up enough trees and maybe someone will actually give you a proper response. The localization crew, at least sometimes, actually responds (though they tend to reply in bursts, and we haven't seen one of those bursts since September).

I'm sure the language gap between English and the dev team doesn't help.

[Emiin]

Since your concerns mainly focus on tooltips being incorrect, you could try posting in the localization forums. Bark up enough trees and maybe someone will actually give you a proper response. The localization crew, at least sometimes, actually responds.

I'm sure the language gap between English and the dev team doesn't help.

In regards to this - in most cases it's not a localization issue that is the problem - it's a simple making sure the numbers on the tooltip are correct. the 330/300 listed on the tooltips is incorrect, not the wording.

[Raldo]

In regards to this - in most cases it's not a localization issue that is the problem - it's a simple making sure the numbers on the tooltip are correct. the 330/300 listed on the tooltips is incorrect, not the wording.

Oh, I'm fully aware of that. That's why I said, "Bark up enough trees and maybe someone will respond". If you're not having luck in one place, try another. It may not be the right department, but it can't hurt.

[Kitfox]

Since your concerns mainly focus on tooltips being incorrect, you could try posting in the localization forums. It's definitely not the right place for it, but bark up enough trees and maybe someone will actually give you a proper response. The localization crew, at least sometimes, actually responds (though they tend to reply in bursts, and we haven't seen one of those bursts since September).

I'm sure the language gap between English and the dev team doesn't help.

The concerns are actually not mainly focused on tooltips. They're just our only official source of information how things should work and that's why they're framed that way. When the tooltip and in-game mechanics don't match, something is wrong somewhere. Both can't be right in these situations. Obviously there are some more underhanded ways to find out about things under the hood, there are limits though, but those are not methods we can use within the Terms-of-Service. However, those methods should be available for Square-Enix and they have the source code, so it should be much easier for the devs to spot the bugs. But alas...

It's good to know about more venues for reporting. We already try to do reports in multiple languages but it's quite hard to find people interested enough in all languages for all jobs. I do think there very well might be a translation issue going on somewhere in the reporting process because the responses from the different language forums are quite different. For example, the German forums are very personal and active in their replies, sometimes even reaching out to you in game to tell you about the process. The English performance in comparison is appalling. The volume of reports for the English side is also much bigger, however, so they probably have less time to be personal. I do wonder if we need to look for our own private translators and do our reports in Japanese, because many of these more obscure bugs can be technical and the finer details could be getting lost in the translation process.

There's absolutely no excuse for the devs to be unable to reproduce steps that any player can, in most cases in just a few minutes, if the reproduction steps are conveyed properly. Often times it seems like the responsible team didn't even try, which is the main issue. It's fine if the bugs are deemed too minor to be fixed or if takes a while for them to get around to it. But it just sucks when you put hours of work into gathering data and documenting it clearly, only to be dismissed by a sloppy process and left wondering if they actually tried it out properly. Do they need more information? Is it not important enough? Do they even know what they're doing? Where's the problem?

[Emiin]

Emiin Vanih here, personally one of the main contributors to the information provided in regards to the Upheaval bug reports, as well as one of the dudes who did work on the 4.2 Warrior changes and sims that are available. I've been pretty vocal in Discord chats and Reddit forums about the disconnect between development and players with the time to figure their game out, so I'll step in here as well.

In regards to the Upheaval bug, it took myself and 2 other friends about 3 hours to verify the data presented, and show that it was a clear indicator of either incorrect tooltip, or (if the tooltip was supposed to be accurate) incorrect damage values. In every situation we did it's 100% reproducible, and the concept that Upheaval is working as intended in either way is somewhat insulting, to say the least. There is clearly something incorrect about the ability, and instead of addressing it, the bug was moved to "Working as Intended," or "Unable to Verify."

The bug is incredibly easy to verify just start hitting a striking dummy once and it's verified. If it's working as intended, then something is clearly incorrect about the ability. Whether it's regarded as a bug is a different matter, but it should be addressed as such. There is even a section if this isn't considered a bug, called "Not a bug report." But this clearly is not the case, and they're clearly ignoring certain complaints that they may find too minor to care about.

-----

In regards to your 3rd option, I'd like to take a moment to sit and talk about how inefficient their balance/job team is. As most people are aware, I'm one of the Theorycrafting nerds that resides in The Balance with my armchair warrior spreadsheets following the developments of this game. On the release of 4.2 and the skill information on the changes of Warrior, I released a Reddit post discussing the changes to 4.2 and how it was a clear DPS increase by almost 5-10% over their then-current status. It took myself and my fellow theory nerds about 3-4 hours of number crunching to realize what the changes would be to Warrior, and how it would lead to a hard disconnect between Warrior DPS and the other tanks.

I bring this up because in the Live Letter at Fanfest, there were two issues brought up that I readily had research arguing against the official Q&A responses (There is no official written transcription of this that I can find) -

- Warrior, and as an extension, tanks, deal too much damage in relation to DPS
- Reverting back to HW scaling on AP would cause a DPS increase.

Warrior doing too much DPS was a fault in their own design from released in January, almost a year ago at this point. I knew this before the patch was released, and in the link above provided math supporting this statement. Warriors already pulled ahead of their tank counterparts. Why did Square-Enix exaccerbate this change? In the patch-notes surrounding 4.2's release, there were multiple Live letters regarding increasing Warrior's DPS output and "usability," furthering the point that they were self-creating the issue that they now cite as a problem.

They created the issue years ago at this point, with the STR addition to accessories. Many of the issues were all knee-jerk reactions to player complaints. Why weren't any of them properly researched?

-----

In regards to the HW scaling, I simply looked at my STR/VIT, subtracted the 51 x 5 STR on my i400 accessories, and multiplied them by 0.45, showing a clear AP decrease. Before anyone cites Food vitality or Weakness as an issue -

- Change vitality bonus on food to a raw 4% HP scale, and it'll be in line with the current food bonus while keeping food DMG increase static.
- Change weakness to 25% damage/healing decrease, instead of stat decrease.

These are simple changes, will result in a DPS decrease for tanks across the board, as well as a further decrease to the effectiveness of Potions on tanks, decreasing their DPS contribution even further.

---

I'd like to note that I found this information out while I was bored staring at google spreadsheets for at most, a few hours. I wasn't paid to do this, and checked this while bored. Verifying this information is not difficult, time consuming, or stressing. I do theorycrafting because it's fun, and in my free time. In my free time, I find issues with the balancing of this game almost immediately. They're being paid to do this and keep messing it up.

[Nemekh]

After waking up and finding out that this thread got moved to "Insufficient Information", perhaps it is time for a video -_-

Edit: New thread here: http://forum.square-enix.com/ffxiv/t...34#post4829834

When I've woken up more, I'll better phrase what I want to say on the matter of the current disconnect between players and SE. The current state is pretty disappointing.

[DRKoftheAzure]

MNK's Shoulder Tackle has Steel Peak's animation and vice versa(I have played another MMO that has a shoulder tackle attack and flavor text says it's a shoulder tackle), and apparently OCD is not important enough to fix animation errors like that, and they can't even address clipping issues properly and they ended up re-purposing 3 DRK action animations for the HALF ASSED ROLE ACTION SYSTEM (yes I am still pissed off about that), so yeah not surprised from this thread... They don't even send an in-game message or mail reply saying that they passed on the suggestion to the dev team...

[Deceptus]

You're not wrong. I've submitted plenty of bug tickets that were clearly bugs and weren't accepted.

But what can you expect from such a small, indie company.