IT folks, realistically what can SE do to alleviate the current situation?

[say0]

They could have upgraded their infrastructure in advance, or subbed it out to AWS or Azure. If they used AWS or Azure they literally could have deployed more servers / resources with a minimal amount of mouse clicks. The incompetence of the current situation is staggering when it could have and should have been preemptively mitigated when they knew full and well they had hordes of new players.

Someone will come along to tell you how its IMPOSSIBLE to utilize temporary cloud servers to lessen the load on their garbage infrastructure. Just wait 5 minutes.

[stu4355226]

Dropping players queue spot is definitely the issue.
From developer perspective, they have the feature to save players spot when the package drops and get kicked out with 2002 code.
But this feature also has time out.... once you're not able to log back in time. you have to re-queue.
Which make you have to set in front of the computer and make sure you have to log back on as soon as you get kick out with 2002 at random time. And queue time is HOURS.

There are definitely ways to scale up or out. but by the time devs finish their work/pass QA and ready to push out the changes..... everything should back to normal.... lol what's the point of doing it?

[Myrany]

Let players who are willing to accept higher latency temporarily transfer to other regions for free. For example, some NA and EU players could temporarily play on JP realms where they can play during Japan's off-peak hours. Likewise, some Japanese players could temporarily play on NA or EU realms during our off-peak hours.

That is only a marginal solution. I am a NA player and I already have a character on a JP server. I can't get far enough into the system to choose a datacenter to play on. I am 2002 long before that point over and over.

[Dyvid]

They have done an amazing job. They have provided compensation, apologized, they sent out a letter in advance setting the expectation that there would be queues like no one has ever seen and the reason for it. They sent out letting players know when peak times are. I have gotten on every day as has most of my FC and friends as we knew if we wanted to get in we do it when others are not.

As consumers of a product there is a number of people not doing things to help and that's where I see a big failing right now. Play when the lines are short and help balance the load. If you can't get on during low volume wait till another time. Don't stress yourself out and be a better person.

I'm trying to cancel my Endwalker Digi Order until the issues are resolved but can't because I used the pre-order unlock code many many months ago. So no, they aren't doing everything.

[shibeouya]

Lots of people seem to be conflating "server capacity" issues with the login queue kicking y ou out. These are 2 completely different things.

Server queues are expected on launch. The queue randomly kicking you out is not.

I have a decade of experience in the industry and worked on very high traffic systems much higher than ffxiv so let me give my perspective:
- Their packet loss explanation makes zero sense. Unless they mean their server loses the packets in which case it's not really a packet loss problem.
- Possible fix #1: add retry logic to the client. If error 2002 or such, try a few times. Low cost and should solve a good chunk of the issues but doesn't get to the root cause. Would require a patch.
- Possible fix #2: tune and configure their login DB so it doesn't drop a random connection. I don't know enough about their infra, but dropping random connections is a common problem and can usually be mitigated by proper tuning. Might take a few days from a few weeks, not trivial, not sure if they have the expertise in house since it's peripheral to the game.
- Possible fix #3: Proper distributed queuing system implementation. Well-known problem with standard solutions. However building that and integrating in their infra is not trivial - we're talking weeks at best or months. Would likely solve all the issues.

In short I'd recommend fix #1 as a band-aid while they work on longer term solutions.

[Hasrat]

They could have upgraded their infrastructure in advance

Look, mate, I'm not even in the field, and even I understand that doing anything "in advance" for a company in this situation requires far more "advance" than a few months. Making proposals, getting approval, sorting out budget allocation, planning, acquisition, installation, and who knows what else.

And you know what? They did do all of that. It's why they've got an entire data center waiting for myriad delays to clear to get up and running. Anything else they could've done would've necessarily had to have happened after OCE was in operation, and, well, since that's stuck waiting, everything else they had planned is also being set back as well.

[Red_Panda]

They can explain why we get kicked from the queue and why they aren't able to code a proper queue system that remembers your place in line. Oh and why does the damn game have to close when you get kicked out of the queue? Why does it not just go back to the title screen like any other proper online game does and all previous expansions did as far as I recall? Why do I have to go through all the hassle of putting in my authenticator every time I get 2002'd?

[kadriel]

Fix the queue. I would be happy if I could actually get in line and actually log in instead of getting an 2002 halfway everytime. It is not that there is a line. For me is that I just can't log in

[Somnolence]

They could have upgraded their infrastructure in advance, or subbed it out to AWS or Azure. If they used AWS or Azure they literally could have deployed more servers / resources with a minimal amount of mouse clicks. The incompetence of the current situation is staggering when it could have and should have been preemptively mitigated when they knew full and well they had hordes of new players.

If only it was so simple. Remember character data is on their databases. So they would need to implement mirroring solution between public cloud and their datacenters which is a nightmare due to latency, bandwidth, security, VPNs etc.

[TristanX]

Compare number of active players from year to year, add servers to help with the load accordingly. Solution to shortage is buy older hardware. May need to add network bandwidth as well.