Letter from the Producer LIVE Part X: Will the netcode issue be honestly addressed?

[Astarica]

I can't give a definitive comment on the technical side, however what I know is that this game was more hacked so far than any other MMORPG I've played in the last 10 years, including a few crappy F2P's made by small companies.

Well teleport is basically impossible to stop because the server views all player movement as teleportation. The other stuff seems to be serious exploits, but not actually hacking, and that also highlights the fact that most serious problems with security are exploits, not hacks. You'll hear far more issues that are equivalent of just someone logging in as admin/admin, as opposed to someone doing some crazy hax0r skills to somehow get past all your layers of security. I remember reading one of the hacks that lets you login as anybody in RIFT because they've some backdoor login protocol, and why would you even build a backdoor login protocol on the client in the first place? Why would anyone running a client version of any MMORPG even need the ability to log into anyone else's account without a password? Yet it's these dumb decisions that are usually the biggest security threats, not because your encryptation wasn't strong enough.

[Alcyon_Densetsu]

Just out of polite curiosity...what exactly do you want to hear from them?

Here's the story.
Basically, if you read my OP, this thread started as an open letter to emphasise the importance of the issue, it was a plea. These two threads are the raw player feedback I wanted to capitalise upon in order to try and get SE's attention, by showing respect but determination in my open letter:
Thread: Sorry SE, but endgame won't work if you don't fix something.
→ Early feedback
Thread: .3 Second Positional Update Delay (Requesting Dev Response)
→ More specific feedback

This thread of mine, which was created 10 days before the "Letter from the Producer LIVE Part X" was meant to try and get SE to answer during that specific event (it's due for next friday). As I said in the OP, I didn't want to divert attention from these existing topics, which is why I keep referencing them all the time, and why this thread of mine is meant to die after this Letter from the Producer. It's just a basic attempt at gathering people and try to make SE react, do or say something.

However… then came Sinth's post. He posted here then made a thread of his own to further expose his take on this.
Thread: Why the netcode issue exists, and why it cannot be fixed.
→ Technical considerations

That's when we learned that clearly, the issue was probably much deeper than we thought.

Which prompted this realisation, at least as far as I'm concerned (post #77):

My hopes hit a low as well after understanding the nub of the issue thanks to Sinth's explanation. As someone said in his thread, SE's facing a dilemma where both ways are bad PR: either admit they failed to take into account the huge discrepancy between Japan's outstanding connections and the rest of the world which is clearly not as well connected, and much more distant from the datacenters; or maintain the silent treatment and face some kind of exodus as people become first aware, then jaded from vainly trying to avoid events that already happened when their screens display them…
Read more of my blabla about how I would personally try to proceed, from what little I know about this situation (not an insider at SE, obviously)

Which means I totally understand the mess they might be in, and the fact it's highly unlikely we get an answer anytime soon.

For what it's worth (post #82):

[…] SE's not particularly known to be quick at responding to feedback or criticism, especially when it's founded on very real considerations; and somehow I can't help but think it has to do with this company's internal management (culture, practice, higher-ups mentality). Whatever Yoshi-P thinks about it, I'm not sure his hands (or rather, mouth) are free in this matter.

Which is why I'm not quite sure we'll hear anything before some time, probably not in 2013, possibly not before they have a working solution already well advanced in the pipes. If Sinth's estimations are correct, that might not be late into 2014, provided they start working on it right now (or already are).

So, how do I want this to proceed?
Well it's not really my place to say, since I don't speak for the community, I simply try to speak to the best of my ability to further this cause but… yeah, well, as I said things may be worse than we thought.

If Sinth was wrong, entirely or partially, and SE could actually fix this is in a timely manner… I would be nothing but happy. Overjoyed. Ecstatic. (no offense dear Sinth, but I think you wish you were wrong too given the situation ^^). But alas, I don't really think they can, or they would have by now. At best, should they shove the millions and the man-hours, maybe for 3.0. Maybe never. I don't know. I wish I knew (hence this thread continues, so far).

I wish they told us:
"Guys we are so sorry about this issue, we know it's problematic for a lot of you. We sincerely apologise for taking so much time to answer, blablabla."
That would be a great first step in regaining trust. Owning the mistake, which a lot of us would understand, I suppose. Would these words, just the words, make us stay? Probably not all of us who are feeling this netcode issue as game-breaking, but probably more than if they said nothing, and above all it would make us consider SE in a different light—a more positive light. I would consider FFXV for instance, right now as we speak my interest couldn't be much lower (and I'm an old FF fan).
But that would still be a blow (again nothing knew, it is already one as we speak), so yeah it would need to be followed by substantial words like:
"But we have this super-plan, and we're going to turn the tables and do this, and this, and that, and it's going to be great, please look forward to it."
I would actually look forward to that.

I could give you my ideal "plan" as a PR-guy (I was actually thinking in my shower today how I'd deal with this if I were them lol) but I'm not sure that's what you want to hear, since it would be filled with conjectures (don't have the numbers etc), so we should probably wait for SE to say something.

All I know is that there's almost always a way to turn a mistake into an opportunity, and Square does have a strong backbone, so they could definitely do something should they really wish to. Maybe that would hurt profits for a while, I suppose, but what's a Q or two without profits so long as you don't sustain losses, and you gain so much trust and respect from your fans, your players, your customers? In the long run this is an event that will stick, and particularly since this is a MMORPG, so I'd wager there is as much to lose as there is to win, should they adopt the wrong or the right stance about this. I see a few ways for the high road. I hope they can too.

(Again, they're in a bad situation but many companies have been there, and have recovered only to shine more. And I'm not talking about spinning this with words, I mean actual, substantial work and 'kindness' towards your customers to make up for your mistakes. Kinda like they already did for ARR actually. Honestly, thinking of all that work, I do feel for them, because I'm sure they meant well, they just… screwed up. Which happens. Probably too often these days at SE, but still. They do show good will.)

Not saying anything however would clearly mean, to me, in corporate-language, that the issue isn't going away before late into 2014, if ever. It would essentially confirm our biggest fears. And I hate that notion. But that's reality for us.

As for waiting, personally I don't mind at all. No MMO announced for 2014 interests me much, and anyhow as a old FF fan I could wait 6 months. Even a year. I really don't mind. You know, you don't follow a franchise for 20 years to refuse a few months wait in order to get the next great instalment. I've actually already stated somewhere in these forums that I would have been fine with waiting 6 months more for the game to be released with more content, so… personally I'd much prefer a great product later than a broken one now.

Does that answer your question? (sorry if I was too long. I do that ^^; )

[Astarica]

I've never heard of a game that successfully ignored a problem and made it go away. If anything refusing to address the problem is likely to make it far worse because everyone will come up with their own crazy theory as to why the problem isn't addressed. If SE just said 'this problem cannot be fixed' (which is very likely the case), at least people will know to move on and probably leave without too much ill-will. Yes you can say that seems like a suicidal statement to say, but do you really think there are people who don't like that this lag that is saying, "Well SE never talked about lag so clearly it's just my imagination'? No people who are bothered by this problem is likely going to be more angry that the problem is simply never addressed as opposed to just being told there's nothing that can be done about it. The lag is a very big issue. There's no way to just pretend it's not there.

[Alcyon_Densetsu]

I've never heard of a game that successfully ignored a problem and made it go away. If anything refusing to address the problem is likely to make it far worse because everyone will come up with their own crazy theory as to why the problem isn't addressed. If SE just said 'this problem cannot be fixed' (which is very likely the case), at least people will know to move on and probably leave without too much ill-will. Yes you can say that seems like a suicidal statement to say, but do you really think there are people who don't like that this lag that is saying, "Well SE never talked about lag so clearly it's just my imagination'? No people who are bothered by this problem is likely going to be more angry that the problem is simply never addressed as opposed to just being told there's nothing that can be done about it. The lag is a very big issue. There's no way to just pretend it's not there.

I agree.

Let's consider Yoshi's recent posts on a variety of topics.

• The Operation and Splitting of Patches for FFXIV: ARR
• Upcoming Changes to Allagan Tomestone Acquisition
• Planned Changes to Warrior in Patch 2.1
• Crafting and High-Level Items

The man is eloquent as ever, to the point, quite honest and transparent in how he approaches players… I do like that, very much so.

Reading such posts, as previous ones, there's just no denying that he knows MMORPG's and has quite the experience in spotting design flaws and features limitations. His observations on past/existing MMO designs are quite spot-on and undeniably true; and if I don't necessarily agree (or rather, have fun) with all the solutions he comes up with, that's more due to a matter of taste (subjectivity and opinion) than a matter of being 'wrong' or 'right'.

That being said, I therefore just can't wrap my around the fact that he wouldn't see the problem with the network code. If he did take the time to at least try his own game while travelling to the US or EU, he just wouldn't fail to noticed something. He actually hints about it in these recent posts, twice.

In "The Operation and Splitting of Patches for FFXIV: ARR":

Now, I’m aware of the requests to release minor tweaks first—improving existing functions and adjusting jobs, for instance. However, most of the functions being discussed are closely tied to the server code, and requires more work than it seems."

About development branches in the same post, he also says:

There are even more test lines than above, but it would get overly complicated, so I won’t touch on them.

One of these test lines could very well be dedicated to a new attempt at netcode, couldn't it?

Also in "Upcoming Changes to Allagan Tomestone Acquisition":

I realize patch 2.1 was originally scheduled to be released in November, but due to problems with server congestion, and the need for server and system improvements, we were forced to delay the release one month. However, I assure you that the wait will be worth it, as the upcoming patch will not only include plenty of new content, but also improvements to the user interface and additional system updates.

These two posts of his are unrelated to the netcode issue per se, but the bits I've highlighted (in teal bold) could not, by any means, have been written by him without thinking of the ongoing netcode uproar. These are quite 'taboo' words these days in SE's communication towards FF XIV players, they make one's mind 'tilt'.

Maybe I'm reading too much into it, maybe not—I wish I knew. If that was a corporate double-talk to hint at the fact that they're aware of some shortcomings of their 'server code' and the need for 'system improvements', beyond the basic bread'n'butter of running any MMO, then this may be it. This may be all we get worded about this issue.

Perhaps he was paving the way for a more substantial update on the netcode issue, so as to be able to deny not having talked about it somehow, yet also deny promising anything about it at the same time. That's a long shot I'm making here, I know, it's anything but far-fetched; but that's common PR subtlety: vague words, mere hints can cover up huge realities. It's all in-between the lines, sometimes.

Anyway, I'm fairly convinced that these two facts are true:

• He knows. He's not the kind of man self-complacent enough to ignore neither the amount of negative feedback, nor the issue itself.
• If he can possibly do something about it, he will. The question are "can he?" and "when?", if ever.

Now, the "Letter from the Producer LIVE" events are quite visible and aimed at hyping up the game, so given his recent posts that directly address several player concerns, I wouldn't hope for any strictly negative insight ("sorry we made a mistake" etc.), it's just not appropriate for this kind of communication, marketing-wise.

Addressing this issue in such a publicised event should imply taking the high road ("sorry, blabla; but we have a great plan to improve it and it will happen soon, please look forward to it etc."). And indeed, it would be the ideal place/moment to make sure all players know that the netcode is addressed and will get better.

We should stay tuned. If a confirmation that they've heard us is coming, it will be tomorrow. If they don't have anything in store—yet, or ever—to make the situation any better, then we won't hear a thing about it, and that (silence) will be our (negative) answer. They know we won't stay tuned eternally, and the clock is ticking; so even if their solution may only come with 2.2 or later, they know it must be addressed sooner rather than later with us players.

[Billie21]

[extrapolations on possible hints droped by Yoshi P about the delay issue]

I so hope you are right. But if it was being worked on, I really can't see any reason why they wouldn't adress it explicitly. Fear of driving away potential players who would take it as a "I'm a non JP player, I will get lag, no point in trying the game"?
The only other reason I can see is that they don't know yet if they will be able to fix it. Even then, saying "working on it, not sure it's possible" would most definitely apease at least part of the players, and go a long way in restauring trust in those that have lost it/are losing it, and it wouldn't even bind them to a promise they might not keep.

[Alcyon_Densetsu]

I so hope you are right. But if it was being worked on, I really can't see any reason why they wouldn't adress it explicitly. Fear of driving away potential players who would take it as a "I'm a non JP player, I will get lag, no point in trying the game"?

Yeah, possible.

⬇︎ Click for more blabla

Actually, that's something I came across quite a bit when I did a bit of community management back in the day (indie web-based games), and later on also as store manager in customer relationship and brand image (cloth retail).

In my personal experience, Producers and Higher-ups have a "brand approach": they would see PR/marketing as the primary sale factor, whereas the technical aspect of the product just needs to be "good enough". I translate that vision into: "sky's the only PR/marketing limit" (looking for the best, always, but there is no 'best') whereas "tech & quality must meet a threshold" (looking for that threshold, anything above is a waste of time/energy/money). See what I mean?

On the contrary, for Directors, Designers, Factory leaders, 'tech people' in general, it's the exact opposite, they have a "product approach": "sky's the only quality limit" whereas "PR/marketing must meet a threshold" (but anything above is useless if quality isn't there etc).

Image vs. substance, in a way.

It's a clear duality, granted a bit cliche and oversimplified as I present it here, but nonetheless quite true (again, in my personal experience) when it comes to hard budget and wording (or silence) decisions. Especially observable in how these leaders drive their teams, value their respective feedback, and report to their own hierarchy (CEO's, shareholders, people like that).

What this means is that these two profiles have different fears, different demons. Producers fear that the image suffers, notwithstanding the reality of the product; Directors worry that the product just isn't good enough, not necessarily in relation to the actual image it actually has. I don't know how Yoshi resolves this duality as both Producer AND Director. Puzzles me. He certainly doesn't seem fearless nor overconfident, so I suppose he's got this cool-yet-stressed typical Japanese feel (see how often he talks about "player stress" and his constant need to justify his decisions by seeking for "player enjoyment", that says a lot about his personality I think; it seems to me that he feels a bit stuck in-between two worlds).

I don't know how the Final Fantasy Council (don't remember the exact name, that special board SE recently appointed to deal with the IP) factors into all that.

Maybe it's the Japanese way, maybe not, but I see a lot of hesitation and carefulness in how SE communicates these days, not to say fear and doubt. Long gone is the Blizzard-like confidence (borderline arrogance).

I think it was a tremendously good decision to fuse Producer & Director to rebuild the game ASAP and coherently, but now I think this game needs to go back to a standard model, because these are two very different positions, profiles, jobs. There needs to be this dialogue, which can't happen in the mind of a single individual. I think it's too much for a single pair of shoulders, considering the magnitude of both a MMO and FF.

So yeah, fear. I sense a lot of fear. Which is not good in the long run because fear usually either makes you freeze, doubt, or reckless as a manager. Revamping the netcode or not? That's such a huge decision, consequences will be shattering no matter which way ARR goes.

The only other reason I can see is that they don't know yet if they will be able to fix it. Even then, saying "working on it, not sure it's possible" would most definitely apease at least part of the players, and go a long way in restauring trust in those that have lost it/are losing it, and it wouldn't even bind them to a promise they might not keep.

I agree. Better temporarily part on good terms with some players who would nevertheless see the game's future in a positive light, than lose players out of frustration because they probably wouldn't ever think of looking back.

It'd be nice to think the parts you've highlighted are some kind of secret wink at the subject, but even mashing the contexts together like this, I'm not seeing it.

In the first quote, he's talking about development branches and testing things. He mentions server code because, most likely, they only have a handful of test servers running various branches of testing code. If a particular feature or function is coded on the client, it's (comparatively) easy for a developer to change some things, compile a version of the client with just those changes, and see what happens, without having an effect on anything anyone else is working on. If a feature is something that needs to be changed on the server, that change needs to go through more steps, and it's less likely any given developer can just whip something up and try it on the fly.

The part about the need for server improvements was specifically talking about the enormous clustercuss that was open beta/launch, and the scrambling they had to do just to get to the point where everyone could even log in and stay connected. That took so much time and resources that it screwed up their schedule. He's not saying anything about the current state of the servers, just that they were badly broken, and that pushed back all their other plans. And "additional system updates" is so laughably vague, I wouldn't take it to mean anything at all, especially given the way SE publishes & translates changes to their games ("Such and such has been adjusted" - Gee, thanks for the helpful patch notes, guyz). In that sentence, "system" just means "not UI," since he mentioned UI changes specifically, but didn't want it to sound like 2.1 was all UI updates.

I'm sure he is aware of the netcode issues. And I hope he does say something on the subject during the live letter (if not before; you never know!). […] But I really don't think anything he's said in these posts has anything to do with it.

I don't know. You do make sense, that's objectively true. I guess I'm just fishing for clues, honestly.

When I first skimmed through the posts, I didn't read much into it; their proper meaning is evidently what you describe. Later on, as I was writing something, I remembered the allusions I quoted above; so I came up with this long-shot of an interpretation.

About patch notes: OMG THANK YOU. Think we should send them a note on what these actually look like?

And subordinate to that hope is the hope that anything he says isn't just more useless victim-blaming (research harder, WARs; zoom out).

HUH! No brainer, indeed.

After my plea you can imagine how that would just dishearten me—to put it mildly. Perhaps beyond return.

[Bixby]

About patch notes: OMG THANK YOU. Think we should send them a note on what these actually look like?

It's a bit late in coming and totally derailing this thread, but this reminded me that I've been meaning to ask about it for every live letter. I'm really just very curious about the reasoning behind it; it doesn't bother me as much as it might seem to.

Regardless, yeah. I really do hope he says something. I'm trying not to be cynical about it. But it's so difficult. I'd rather keep my expectations low and hopefully be pleasantly surprised than expect something and be once again confounded. It's such a weird duality; Yoshi is obviously much more in-touch when he makes the larger posts and the non-live letters, and he seems to "get it" about so many things. But then you get those two statements in particular (research more, zoom out), that are just... mind-bogglingly out of touch. And mind-bogglingly poor form, too.

[Bixby]

teal bold

It'd be nice to think the parts you've highlighted are some kind of secret wink at the subject, but even mashing the contexts together like this, I'm not seeing it.

In the first quote, he's talking about development branches and testing things. He mentions server code because, most likely, they only have a handful of test servers running various branches of testing code. If a particular feature or function is coded on the client, it's (comparatively) easy for a developer to change some things, compile a version of the client with just those changes, and see what happens, without having an effect on anything anyone else is working on. If a feature is something that needs to be changed on the server, that change needs to go through more steps, and it's less likely any given developer can just whip something up and try it on the fly.

The part about the need for server improvements was specifically talking about the enormous clustercuss that was open beta/launch, and the scrambling they had to do just to get to the point where everyone could even log in and stay connected. That took so much time and resources that it screwed up their schedule. He's not saying anything about the current state of the servers, just that they were badly broken, and that pushed back all their other plans. And "additional system updates" is so laughably vague, I wouldn't take it to mean anything at all, especially given the way SE publishes & translates changes to their games ("Such and such has been adjusted" - Gee, thanks for the helpful patch notes, guyz). In that sentence, "system" just means "not UI," since he mentioned UI changes specifically, but didn't want it to sound like 2.1 was all UI updates.

I'm sure he is aware of the netcode issues. And I hope he does say something on the subject during the live letter (if not before; you never know!). And subordinate to that hope is the hope that anything he says isn't just more useless victim-blaming (research harder, WARs; zoom out). But I really don't think anything he's said in these posts has anything to do with it.

[theflo84]

I've never heard of a game that successfully ignored a problem and made it go away. If anything refusing to address the problem is likely to make it far worse because everyone will come up with their own crazy theory as to why the problem isn't addressed. If SE just said 'this problem cannot be fixed' (which is very likely the case), at least people will know to move on and probably leave without too much ill-will. Yes you can say that seems like a suicidal statement to say, but do you really think there are people who don't like that this lag that is saying, "Well SE never talked about lag so clearly it's just my imagination'? No people who are bothered by this problem is likely going to be more angry that the problem is simply never addressed as opposed to just being told there's nothing that can be done about it. The lag is a very big issue. There's no way to just pretend it's not there.

I agree. If i quit now then i will allways have in mind that SE never did anything about the issue and that they where not able to do something about it. That will always be in my mind. Not only when i talk with frinds about FF 14... It will remind me everytime when i see a new game from SE...

[beydan]

Back to the front page with you.