IT folks, realistically what can SE do to alleviate the current situation?

[Issigri]

They could implement ways to identify afk'rs using scripts not to be kicked - from other players being able to tag them to automated internal processes to help identify them. GM's could check tagged players and if they are found scripting to avoid the afk boot then suspend the account for 6 months - no appeal. The vast majority of the players doing this would not risk it and stop. Right now they are easily found and doing something obvious like jumping 3 times every 10 seconds - and this doesn't account for the people inside a house doing it out of sight. Right now by what I read we are only allowed 750 active players per world - and I'm sure a chunk of that is taken up by people afk scripting so they don't get booted because there is no downside if they are discovered other than being kicked by a GM.

[Matrinka]

They could have upgraded their infrastructure in advance, or subbed it out to AWS or Azure. If they used AWS or Azure they literally could have deployed more servers / resources with a minimal amount of mouse clicks. The incompetence of the current situation is staggering when it could have and should have been preemptively mitigated when they knew full and well they had hordes of new players.

Genuinely curious, no sarcasm, but which part of the SE team do you think is guilty of doing this? Business side, Yoshi-P, dev. team, or some other party I just don't know enough to mention?

Now, I know this game is paying SE's bills. It is vital to keep it running properly and keeping the user-base happy and paying. That alone makes me think this is just a combination of getting too popular at a time when the way things work violently shifted. No one (and everyone interested) is to blame... but it certainly stings when seeing advertisements for the game enticing even more people to sign up for the madness. There aren't enough resources for the current base they had, but they're trying to add more into the pool despite the water turning yellow.

[Razed]

If only it was so simple. Remember character data is on their databases. So they would need to implement mirroring solution between public cloud and their datacenters which is a nightmare due to latency, bandwidth, security, VPNs etc.

It should already be in the cloud, the entire industry (and government!) has been moving to the cloud as soon as it started to become viable in the early 2000s. Source: Me, I've worked for IBM, HP, big banks and government and have spent years of my life in datacenters (which I'm not bragging about, perpetual 67F temp gives you innate resistance to cold tho.)

[Hasrat]

Compare number of active players from year to year, add servers to help with the load accordingly.

If only the game hadn't experienced an explosive population growth in the past year, faster than they would've been able to react to under ideal circumstances... Also, y'know, they were already in the process of adding servers, so.... kinda already doing all of this, except the time scales involved are a bit longer than we might expect/like.

[TristanX]

If only the game hadn't experienced an explosive population growth in the past year, faster than they would've been able to react to under ideal circumstances... Also, y'know, they were already in the process of adding servers, so.... kinda already doing all of this, except the time scales involved are a bit longer than we might expect/like.

Yeah, shortage issues. Getting latest hardware takes time. Let's hope SE could get it sooner.

My workaround is queue earlier before the peak hours. Remote desktop just to queue is another way.

[van_arn]

Dropping players queue spot is definitely the issue.
From developer perspective, they have the feature to save players spot when the package drops and get kicked out with 2002 code.
But this feature also has time out.... once you're not able to log back in time. you have to re-queue.
Which make you have to set in front of the computer and make sure you have to log back on as soon as you get kick out with 2002 at random time. And queue time is HOURS.

There are definitely ways to scale up or out. but by the time devs finish their work/pass QA and ready to push out the changes..... everything should back to normal.... lol what's the point of doing it?

In normal circumstances I'd agree, but this time things might not go back to normal because they have no option to increase capacity.

Improving the queue system is the last resort available to Square. The only other way right now for queues to improve is for huge swaths of players to give up on the game entirely, and that is not what Square wants.

[Razed]

Genuinely curious, no sarcasm, but which part of the SE team do you think is guilty of doing this? Business side, Yoshi-P, dev. team, or some other party I just don't know enough to mention?

I don't think it has anything to do at all with Yoshi-P or the dev team, they aren't the ones dealing with the engineering side of the business. To be honest I'm probably being way too harsh with saying it's incompetence, but they are behind the times. It works, until it doesn't in their case. It didn't have to get it where it doesn't work. They definitely should be moving everything to the cloud even if its their own personal datacenter fulfilling their cloud needs where they can spin up virtual assets but personally they probably aught to be farming everything network and server side to a company that specifically deals with offering cloud solutions like AWS (Amazon) or Azure (Microsoft), or somewhere else.

[Feronar]

I have another idea that likely won't be popular, but I think would be the most fair solution: Ration everyone's playtime. Temporarily implement a daily playtime limit of X hours, and enforce it at the service account level. The limit could be increased, then eventually removed as the surge dies down. This would help deter AFKers, as going AFK would still count against your limited playtime. They could also incentivise playing during off-peak hours (I.e. early morning) by giving you an extra hour or two of playtime that can only be used during off-peak hours.

To start, perhaps they could give you 3 normal hours (Can be used any time of the day), plus 2 off-peak hours (Can only be used during off-peak hours), for 5 hours total per day if you are willing and able to play during off-peak hours.

[Vahlnir]

Have players enter death arenas at the que screen for the right to play the rest of the game.

[DPZ2]

They could have upgraded their infrastructure in advance, or subbed it out to AWS or Azure. If they used AWS or Azure they literally could have deployed more servers / resources with a minimal amount of mouse clicks. The incompetence of the current situation is staggering when it could have and should have been preemptively mitigated when they knew full and well they had hordes of new players.

I am constantly amazed that anyone still believes this game runs on Intel or AMD chip-based virtual systems, especially given the fact that they can't seem to get the systems they want. White-box Linux systems are readily available and have been so almost the year round. [Yeah, I work for a VAR who sells Sun/Oracle, Intel/AMD/SPARC/cloud, among other things].

I expect that a Japanese company laying out the architecture for an MMO might utilize RISC-based systems and select a Japanese computer company as their main vendor (Fujitsu and NEC come to mind).

We don't know, and it's pretty futile to even make a guess.

One thing I can guarantee right now. SE does not utilize any of the major Cloud companies for the game's infrastructure. What part of "Japanese/NA/EU Datacenter" don't people understand?