The Disconnect between Players and Developers: Bug Reports Not Taken Seriously?

Hello,

Lynn Nuvestrahl, the Machinist theorycrafter, here again. I'd like to talk about improving the bug report workflow, especially for bugs regarding game balance.

TL;DR: Bug Reports for game balance issues require excessive documentation to be properly investigated and the investigations that do happen are sloppy.

I recently made a couple of bug reports about such issues that were marked as "Unable to Verify" along with today's patch:
-Tornado Kick: http://forum.square-enix.com/ffxiv/t...in-the-tooltip
-Upheaval: http://forum.square-enix.com/ffxiv/t...e-is-incorrect

I find this disturbing because I went to great lengths in both of my reports to very clearly indicate how to reproduce and observe the effects of the bugs. For those unfamiliar with the "Unable To Verify" category, this is the description:

Quote:

---

Originally Posted by Gildrein

Bugs that have been reported but cannot be replicated even though they have been investigated and tested will be moved here.

---

These are bugs that have been confirmed independently by multiple players. Furthermore, the Upheaval bug is not even the first time it has been reported, the first time being almost a year ago (details: http://forum.square-enix.com/ffxiv/t...en-in-Defiance). When the bugs can be confirmed even by untrained people simply by observing the game, it shouldn't be that hard for devs to catch them.

Especially in the case of the Tornado Kick bug, with current rotations MNKs can gain hundreds of potency per minute by properly utilizing the skill. If it has the wrong potency, it's a bug that actually breaks the job balance, as most buffs and nerfs are within the 1-2% dps adjustment range and this is most certainly well within that. Upheaval is a less severe example since most of the time tank stances are not heavily utilized in endgame, but nonetheless, the difference of hundreds of potency should be easily observable.

It's important for game balance bugs like these to be tested carefully because there's only a handful of people who even notice these things, and without proper investigation they never get fixed. A good example of this is the MCH Flamethrower bug. The skill was bugged since Stormblood launch, presumable for about 10 months. It was fixed but only after it was reported multiple times through different channels. Here's a couple of them:
-http://forum.square-enix.com/ffxiv/t...or-Direct-Hits
-http://forum.square-enix.com/ffxiv/t...-not-Crit-Dhit
-http://forum.square-enix.com/ffxiv/t...tisch-Treffen.
The bug report was not accepted until this particular report that stacked up excessive proof about its existence:
-http://forum.square-enix.com/ffxiv/t...-normal-damage

Now, I can think of a few explanations for the problems:

1. SE is too busy to deal with the bugs because of the expansion and they were pushed aside. Fair enough. Why weren't they moved to "Confirmed" or kept in "Accepted" category until solved?
2. The Bug Report team is ill-equipped to deal with these sort of bugs or there's a disconnection in the report pipeline that doesn't allow these bugs to be properly checked. Can the workflow be improved to better account for this bug type?
3. The operational team working on bug fixes actually doesn't know how their own code works or is supposed to work.

I truly hope it isn't the 3rd option, but it wouldn't be unheard of as we've seen a couple of times players coming up with more efficient skill rotations than the dev team has expected, here for example:

Quote:

---

Originally Posted by Gildrein

In my previous answer I mentioned that we check DPS against unmoving targets and balance jobs while considering the ideal action rotation. However, everyone has put in great effort studying ninja and has come up with a rotation that is slightly more effective than what we had imagined allowing for a slightly higher percentage of DPS.

---

Please Square-Enix, it takes hours to document bugs properly like this and seeing them go to the pile of unsolved issues without proper investigation is such a slap in the face. Is there a problem in the bug report processes to deal with these sort of balance bugs? What can we do to make it easier for you address these issues? Do we really need to mobilize dozens of people every time to be taken seriously?

I'm honestly running out of ideas at this point when spreadsheets and graphs no longer work for you. Especially in the light of the recent Ungarmax scandal I find this very depressing.

Many bugs that make it past launch that aren't "critical"usually get placed into a bucket that's basically labelled "fix if you have time". It's very easy for these issues to get completely forgotten about by both the dev and qa teams, especially if the issues are not kept up to date regularly on their internal systems or they might get backburnered unless there is a surge in CS reports about it (i.e. ungarmax exploit).

Or it could just be that the issue is irrelevant due to future changes that are coming, at fanfest Yoshi mentioned tuning each job again for 5.0. Might be that issues with certain skills are being slated to be fixed, modified, or removed during that time. Why spend dev time fixing a minor issue, that in 6 months won't even be relevant?

As far as the category change, I'm not sure. That'd be something solely up to the devs to reveal, but don't think I've ever seen a dev officially post on the forums aside from messages from Yoshi and Koji.

Could also be that the issue has already been fixed on the version the devs/qa is on, so they weren't able to replicate it on their end.

Since your concerns mainly focus on tooltips being incorrect, you could try posting in the localization forums. It's definitely not the right place for it, but bark up enough trees and maybe someone will actually give you a proper response. The localization crew, at least sometimes, actually responds (though they tend to reply in bursts, and we haven't seen one of those bursts since September).

I'm sure the language gap between English and the dev team doesn't help.

Emiin Vanih here, personally one of the main contributors to the information provided in regards to the Upheaval bug reports, as well as one of the dudes who did work on the 4.2 Warrior changes and sims that are available. I've been pretty vocal in Discord chats and Reddit forums about the disconnect between development and players with the time to figure their game out, so I'll step in here as well.

In regards to the Upheaval bug, it took myself and 2 other friends about 3 hours to verify the data presented, and show that it was a clear indicator of either incorrect tooltip, or (if the tooltip was supposed to be accurate) incorrect damage values. In every situation we did it's 100% reproducible, and the concept that Upheaval is working as intended in either way is somewhat insulting, to say the least. There is clearly something incorrect about the ability, and instead of addressing it, the bug was moved to "Working as Intended," or "Unable to Verify."

The bug is incredibly easy to verify just start hitting a striking dummy once and it's verified. If it's working as intended, then something is clearly incorrect about the ability. Whether it's regarded as a bug is a different matter, but it should be addressed as such. There is even a section if this isn't considered a bug, called "Not a bug report." But this clearly is not the case, and they're clearly ignoring certain complaints that they may find too minor to care about.

-----

In regards to your 3rd option, I'd like to take a moment to sit and talk about how inefficient their balance/job team is. As most people are aware, I'm one of the Theorycrafting nerds that resides in The Balance with my armchair warrior spreadsheets following the developments of this game. On the release of 4.2 and the skill information on the changes of Warrior, I released a Reddit post discussing the changes to 4.2 and how it was a clear DPS increase by almost 5-10% over their then-current status. It took myself and my fellow theory nerds about 3-4 hours of number crunching to realize what the changes would be to Warrior, and how it would lead to a hard disconnect between Warrior DPS and the other tanks.

I bring this up because in the Live Letter at Fanfest, there were two issues brought up that I readily had research arguing against the official Q&A responses (There is no official written transcription of this that I can find) -

- Warrior, and as an extension, tanks, deal too much damage in relation to DPS
- Reverting back to HW scaling on AP would cause a DPS increase.

Warrior doing too much DPS was a fault in their own design from released in January, almost a year ago at this point. I knew this before the patch was released, and in the link above provided math supporting this statement. Warriors already pulled ahead of their tank counterparts. Why did Square-Enix exaccerbate this change? In the patch-notes surrounding 4.2's release, there were multiple Live letters regarding increasing Warrior's DPS output and "usability," furthering the point that they were self-creating the issue that they now cite as a problem.

They created the issue years ago at this point, with the STR addition to accessories. Many of the issues were all knee-jerk reactions to player complaints. Why weren't any of them properly researched?

-----

In regards to the HW scaling, I simply looked at my STR/VIT, subtracted the 51 x 5 STR on my i400 accessories, and multiplied them by 0.45, showing a clear AP decrease. Before anyone cites Food vitality or Weakness as an issue -

- Change vitality bonus on food to a raw 4% HP scale, and it'll be in line with the current food bonus while keeping food DMG increase static.
- Change weakness to 25% damage/healing decrease, instead of stat decrease.

These are simple changes, will result in a DPS decrease for tanks across the board, as well as a further decrease to the effectiveness of Potions on tanks, decreasing their DPS contribution even further.

---

I'd like to note that I found this information out while I was bored staring at google spreadsheets for at most, a few hours. I wasn't paid to do this, and checked this while bored. Verifying this information is not difficult, time consuming, or stressing. I do theorycrafting because it's fun, and in my free time. In my free time, I find issues with the balancing of this game almost immediately. They're being paid to do this and keep messing it up.

Quote:

---

Originally Posted by Raldo

Since your concerns mainly focus on tooltips being incorrect, you could try posting in the localization forums. Bark up enough trees and maybe someone will actually give you a proper response. The localization crew, at least sometimes, actually responds.

I'm sure the language gap between English and the dev team doesn't help.

---

In regards to this - in most cases it's not a localization issue that is the problem - it's a simple making sure the numbers on the tooltip are correct. the 330/300 listed on the tooltips is incorrect, not the wording.

Quote:

---

Originally Posted by Emiin

In regards to this - in most cases it's not a localization issue that is the problem - it's a simple making sure the numbers on the tooltip are correct. the 330/300 listed on the tooltips is incorrect, not the wording.

---

Oh, I'm fully aware of that. That's why I said, "Bark up enough trees and maybe someone will respond". If you're not having luck in one place, try another. It may not be the right department, but it can't hurt.

After waking up and finding out that this thread got moved to "Insufficient Information", perhaps it is time for a video -_-

Edit: New thread here: http://forum.square-enix.com/ffxiv/t...34#post4829834

When I've woken up more, I'll better phrase what I want to say on the matter of the current disconnect between players and SE. The current state is pretty disappointing.

Quote:

---

Originally Posted by Raldo

Since your concerns mainly focus on tooltips being incorrect, you could try posting in the localization forums. It's definitely not the right place for it, but bark up enough trees and maybe someone will actually give you a proper response. The localization crew, at least sometimes, actually responds (though they tend to reply in bursts, and we haven't seen one of those bursts since September).

I'm sure the language gap between English and the dev team doesn't help.

---

The concerns are actually not mainly focused on tooltips. They're just our only official source of information how things should work and that's why they're framed that way. When the tooltip and in-game mechanics don't match, something is wrong somewhere. Both can't be right in these situations. Obviously there are some more underhanded ways to find out about things under the hood, there are limits though, but those are not methods we can use within the Terms-of-Service. However, those methods should be available for Square-Enix and they have the source code, so it should be much easier for the devs to spot the bugs. But alas...

It's good to know about more venues for reporting. We already try to do reports in multiple languages but it's quite hard to find people interested enough in all languages for all jobs. I do think there very well might be a translation issue going on somewhere in the reporting process because the responses from the different language forums are quite different. For example, the German forums are very personal and active in their replies, sometimes even reaching out to you in game to tell you about the process. The English performance in comparison is appalling. The volume of reports for the English side is also much bigger, however, so they probably have less time to be personal. I do wonder if we need to look for our own private translators and do our reports in Japanese, because many of these more obscure bugs can be technical and the finer details could be getting lost in the translation process.

There's absolutely no excuse for the devs to be unable to reproduce steps that any player can, in most cases in just a few minutes, if the reproduction steps are conveyed properly. Often times it seems like the responsible team didn't even try, which is the main issue. It's fine if the bugs are deemed too minor to be fixed or if takes a while for them to get around to it. But it just sucks when you put hours of work into gathering data and documenting it clearly, only to be dismissed by a sloppy process and left wondering if they actually tried it out properly. Do they need more information? Is it not important enough? Do they even know what they're doing? Where's the problem?

MNK's Shoulder Tackle has Steel Peak's animation and vice versa(I have played another MMO that has a shoulder tackle attack and flavor text says it's a shoulder tackle), and apparently OCD is not important enough to fix animation errors like that, and they can't even address clipping issues properly and they ended up re-purposing 3 DRK action animations for the HALF ASSED ROLE ACTION SYSTEM (yes I am still pissed off about that), so yeah not surprised from this thread... They don't even send an in-game message or mail reply saying that they passed on the suggestion to the dev team...

You're not wrong. I've submitted plenty of bug tickets that were clearly bugs and weren't accepted.

But what can you expect from such a small, indie company.

I don't have much to add than been said but I support the OP, I hope it will gain some traction with the dev team and it'll be improved on.

Some of the main problems is that too many people who don't play video games in general and people who only care about getting a big paycheck are some of the main reasons why I can't take "community representatives" and "customer support" and "GMs" seriously... Just look at "game journalists" today, they don't even play video games and are very disrespectful to the point where I wonder if they got kicks out of bullying people a lot when growing up...

Really wish they'd do more than just post if they're closing your post for 'not enough information'/moving the thread somewhere else, or just flat out ignoring it. I had multiple reports a few years ago where I had lost about 13 days of my game time, because I logged out of the Lav Beds, but when I tried to log back in the game would always crash back to desktop. They didn't once care to reply to anything besides a 'not enough information, so I'm closing this' when I gave them every bit of info I could get.

From my experience of reporting bugs, there has rarely been a case where it has actually been an easy process. "Not enough information" being used on what clearly has enough reason. If a player knows what the reason for the bug is and it is clearly stated in the bug report. Why is it then that the team responsible for bug reports are unable to use that information? Ie, if the issue is a clear server side issue and the cause is not affected by anything on the client side, why is there a need to post computer information as well as network information to get it accepted? Or you have issues like Upheaval where the decision for the bug report is inconsistent. First one is that it's lacking information, the second one that it's working as intended and then when reported again, they're unable to verify it. With information provided by the players that clearly shows the issue.

I don't know where the connection between the players and developers break, but it's pretty clear that the reporting system is not doing its job, biggest offender being Ungarmax and while it ws stated something was going to be done about it, there still seems to be a pretty clear case where information gets blocked.

I find it quite strange that the Tornado Kick and Upheaval bugs couldn't be verified, because I made a very similar report about MCH Hypercharge potency (http://forum.square-enix.com/ffxiv/t...he-description). I compared damage ranges, just like in the reports linked in the OP, but in this case they fixed the tooltip. I wonder what makes Tornado Kick and Upheaval different. Maybe it really needs video, like Nemekh said...

Customer service in general is pretty weak when it comes to anything other then maybe select types of harassment reports.

Whether it's due to miscommunication, lack of concern, and/or lack of staff shall forever remain a mystery.

Does anyone know if cross world /tell is actually working like it's supposed to? I'm on PS4 and doesn't seem to be working like it should...

I did send bug report in-game but a GM had to respond to while I was AFK but they way they have the whole bug report setup right now... it's outdated and when people send a bug report on something that should be working correctly like cross world /tell when it's not even working like it should, then clearly I have no hope that world visit is even going to work like it should ESPECIALLY if you add people to friends list and you want to strike up a conversation with them only to find out that the social aspect of the game is not working like it should because no one actually tested this in an actual player environment...

Although Activision Blizzard and EA is an example of game companies not listening to their customers when it really matters in the long run... If World Visit ends up being a flop because of things not working like they should and the GMs don't think that we supplied enough info because they need to know the PS4 model # or something like that when people just want to know if the devs can replicate said bug/glitch using the systems that game is available on...

Quote:

---

Originally Posted by DRKoftheAzure

Although Activision Blizzard and EA is an example of game companies not listening to their customers when it really matters in the long run... If World Visit ends up being a flop because of things not working like they should and the GMs don't think that we supplied enough info because they need to know the PS4 model # or something like that when people just want to know if the devs can replicate said bug/glitch using the systems that game is available on...

---

Oh, every company has certain levels of obliviousness. EA is absolutely horrible and I imagine anyone working for EA must feel utterly miserable from the decisions EA makes. EA doesn't pay QA people for example, they seek people to volunteer to QA at fan conventions and dangle free games over their head. Like, it cost more for a ticket to the convention than any free game you'd get from QA'ing, EA has to be completely unable to see their toes. Their mobile games are shameless wallet pinchers.

Activision-Blizzard decided to wreck one of their leading IP's by ... trying to make it a mobile game? Like far be it for the players to decide what the developer decides to make with their IP (Square-Enix does things like this too), but for Western Developers mobile is the cash cow, but the games made for mobile are utter soulness rubbish and only about putting their hands deeper into your wallet.

Bethesda's big problem is that they either don't QA at all, or their game engine is held together by gum and bailing wire. The game engine (Gamebryo) is the core, and in use since Elder Scrolls III (Morrowind) from 2002. Fallout 3, 4 and 76 all appear to use the same game engine technology as far as I can see, and one look at any page on any of those games Wiki's often links to a half dozen or more bugs, some game-breaking. You simply can't play any of Bethesda's games without hitting a game-breaking bug. My "high end system" in FO4, the pathfinding is utterly broken, and you can basically shoot melee mobs like fish in a barrel just by finding something to stand on that the pathfinding can't "jump" to (the mob will literately run all over the place trying to find a path to you) and you just pick them off with the sniper rifle.) The most brokenish combat is in the Vault 88 DLC, which you can basically just build three turrets and let even the boss mobs come to you. With Fallout 76 (which I haven't played), it's broken even worse than XIV 1.0. Despite that I still feel compelled to finish Fallout 4. I do not feel compelled to even download Fallout 76 because every game Bethesda releases is a buggy mess, and experience from 20 years of MMO games has shown that game-breaking bug filled, MMO games don't last very long before shutting down.

Square-Enix's problem is that they don't solicit feedback in the first place. FFXIV V1.0 is a clear example of a game made by accountants who didn't let anyone talk to each other. It was fun to people who liked the micromangement elements, and had overkill hardware to play it at release time, but that was a small and shrinking number of players. It was too slow paced to be fun, and to shut it down entirely would damage the Final Fantasy brand.


Bethesda and Blizzard should learn from how Square Enix fixed its Final Fantasy XIV problem, by Mike Fischer former CEO of Square Enix America.

Quote:

---

No, really — listen
The FFXIV developers used the harsh feedback provided on outside forums as an independent source of input. This was by no means the only resource used, but it did help inform the decision-making, as did input from all the Square Enix subsidiaries around the world.

---

If they'd at least explain what information is missing when they move it that would be nice.

I submitted a bug report a while back for the Blessed Boltkeep's Needle, which has had an animation bug since HW where it stops glowing when you use any action, as verified by others. I don't know how much more detailed I could have been in that post. I guess at this point they'd only want a video, but I have a very, very bad internet connection and even a video of a few seconds could take hours to upload. :/