Letter from the Producer LIVE Part X: Will the netcode issue be honestly addressed?

[Sinth]

I used to work for several years as a systems programmer in a data center which hosted several MMOGs. Perhaps I can give a bit of insight into this problem.

The issue everyone is complaining about stems from an architectural design decision made by SE in the early stages of the game's development. The decision made was to design the game in a way which makes the server dictate the live-state of the game as opposed to the client. This is contradictory to the vast majority of online games on the market, presently, as well as in the past. I will explain below the two different methodologies.

Client Live-State:
Most of the game actions are driven by the client. However, the entities and environment are controlled by the server. When you give a command to jump, your client sends a command to the server, which acknowledges, and replies back to the client with a response to jump. Once the communication has completed, your character then jumps on your screen. Say, for instance you are fighting a monster. You attack the monster but your network connection is poor. So what happens is: You send the command to attack. There is a delay in sending the command to the server. The monster continues to attack you (successfully), as it is controlled by the server. When your connection resolves, you finally make your attack. This results in the player's experience being - 'I'm giving commands but they are delayed on my screen.' However, if a monster creates an AoE circle, it will be in real-time on the client because the server sends the command "monster A is doing an AoE attack at such and such time" When your client receives this information, it syncs to the time provided by the server, so the AoE may APPEAR late, but your reaction to it will be on time, because the client is in a live-state. So if you step out of the AoE on your client, when it goes off, your client will tell the server "The AoE landed, but the client was out of it's range", so in the end, even if you are lagging a little bit, you can still dodge the AoE.

Server Live-State:
Most of the game actions are driven by the server. When you give a command to jump, your client jumps on your screen, but in the live-state of the server you have not jumped until it receives your command to jump. So if you're lagging, it may look like you're jumping on your screen, but to the server, you are standing still. This gives the player a false sense of what is happening in the game. So how is this any different than client live-state? Well, for things like jumping its not much different, but when it comes to combat and time-intensive actions, its a whole other animal. Say for instance, monster A drops an AoE. The AoE is live on the server but you're lagging and haven't received the info yet (monster appears to be sitting there). As soon as you get the information, the monster appears to do his cast and the AoE circle appears. The difference is, you are out of sync with the server and the server rules all, so even if you step outside of the AoE, if your client doesn't get the command to the server to update your position before the AoE lands, you get hit by it. And that, my friends, is why you all died to landslide / bombs / any AoE that you were out of on your screen.

The reasons why SE would choose server dictated live-state:
-Prevents desync bugs (on the server)
-Prevents countless potential duping bugs
-Prevents (in theory) teleportation and other unauthorized client commands from being accepted
-If the player has a good connection to the server, little to no delay is observed by the player

The detractors of server live-state:
-Poor connections results in nearly unplayable conditions, specifically conditions which require fast reaction time
-Server infrastructure is more costly (servers must do much more data processing)

In the past, server live-state was extremely rare because internet connections were still poor / narrow-band.

Can it be changed?
Short answer: No.
Long Answer: Technically, yes, but it would be extremely costly and time consuming. SE Would lose millions and it would take months to over a year to accomplish.

I hope I explained things well enough for most people to understand. Let me know if you want clarification on anything.

Cheers.

[Alcyon_Densetsu]

I used to work for several years as a systems programmer in a data center which hosted several MMOGs. Perhaps I can give a bit of insight into this problem.
[…]
I hope I explained things well enough for most people to understand. Let me know if you want clarification on anything.

Cheers.

Wow! That is probably the most enlightening explanation I read on the issue. Thank you very much for such an interesting read!

I have a couple of follow-up questions if you don't mind

1. Considering that in Japan, most customers are connected to the internet through very good connections (100Mbps symmetrical with quite low latency), do you think it could explain why SE chose a server live-state architecture? And do you think it's a wise choice considering the quality of worldwide connections as of 2013 (NA, SA, EU, Asia…), bearing in mind that's there's only 1 datacenter for each half of the world?

2. Do you know of any other games that are also based on such a server live-state architecture?

EDIT: Sinth made a thread of his own to make his explanation more visible, something which i wholeheartedly support, therefore I'll post my questions in there.

[Sinth]

Questions

Thanks, glad I could help. To answer your questions:

1a. I would say your guess is likely to be quite accurate as to why SE would choose this model of networking in their game. In addition to the advanced network infrastructure in Japan, the geographical size of the country also provides for a reduced latency due to less lengthy hops a connection has to make between server and client.

1b. I would say it was an unwise decision based on the limited availability of localized servers (2 data centers for the whole planet). If SE had licensed servers to data centers all over the world in order to serve local communities, these issues would not be so apparent.

2. No MMOs that I know of, specifically, for the reasons stated above.

[Alcyon_Densetsu]

For someone who isn't a native English speaker, I've gotta hand it to you: you are amazingly eloquent. I don't think this issue could've been brought up any better than what you just did. So, well done, sir!

Eloquently put, and also good on a fellow community member to explain his speculation on the issue. I do not think this issue will be addressed in the live letter. However if there was ever a well placed plea for a response, I have to acknowledge that this is it.

Thank you very much, both of you, for the very kind comments.

Answers

Thank you for the answers

(Also, sorry for double-posting your topic, I just wanted to switch over there to further discuss this, since you're right making your explanation an OP will make it much more visible to this community)

[Rydiah]

Thank you Sinth! I agree that this is the best explanation I've seen about this issue. My main is Sch, so I'm constantly watching the HP bars. Even when I'd focus the boss, I never got to actually see what the issue was. It wasn't until I leveled up my Drg, I was able to keep an eye on the boss at all times. Doing Chimera in Cutter's Cry, I finally saw it. Sometimes the message about what color his eyes are glowing would pop up, and then maybe .5 seconds later, he'd start casting Ram's voice (or Dragon's voice...whatever, I assume this is the way its SUPPOSED to work) Giving me more than enough time to move. But other times, the message would pop up and the cast bar was already like 10% in.

This issue has given me the push I needed to switch finally switch ISPs. On Monday I'll have much faster internet. If that doesn't improve the issue for me...then I just don't know anymore lol.

On topic. I really hope they address it in the Live Letter, but I'm not holding my breath. I really didn't like Yoshi's latest response...if you could call it that. To me, the silence suggests they may be incapable of fixing it. I just want to be able to see the circle, THEN move...not have to move before the boss starts casting.