Data center travel suspended

[Stormpeaks]

Well this didnt take long

We have received over 15,000 visit requests per minute and confirmed that the servers are unable to keep up with the high server load.

Due to the number of requests which makes it difficult to resolve the issue while keeping the Data Center Travel system open, we will temporarily suspend the Data Center Travel system and gradually open this feature as we monitor the situation.

Housing fiasco 2.0

Also my char is stuck on the other datacenter, so that's fun !

[Kaurhz]

Did you or anyone actually really expect this to end up any other way...?

[Stormpeaks]

Did you or anyone actually really expect this to end up any other way...?

I did expect a little bit more preparation than this yeah, maybe dont release features that are going to be massively used and that will break in the first hour. Internal testing maybe ?

[GeminiReed]

I did expect a little bit more preparation than this yeah, maybe dont release features that are going to be massively used and that will break in the first hour. Internal testing maybe ?

Perhaps they should have some better rate limiters but the volume problem is going to correct itself once the novelty wears off. They release these things in an off-patch both as a volume test and to let that novelty period soak a bit. The remaining unknown will be if the DCTS servers can handle the steady-state traffic.

Depending on how the feature was designed they might even be able to temporarily add hardware to deal with the surge.

[Kaurhz]

I did expect a little bit more preparation than this yeah, maybe dont release features that are going to be massively used and that will break in the first hour. Internal testing maybe ?

Regardless of how much testing they would do, the first thing people are always going to do is dogpile a new feature, and this will always cause congestion to happen, especially when practically everyone and their grandmother dogpile a new feature. Managing that congestion is not very easy to do, especially when the number of requests potentially far exceeds the capacity that the servers can handle. This is basically akin to the community doing a DDoS attack.

[TaleraRistain]

Regardless of how much testing they would do, the first thing people are always going to do is dogpile a new feature, and this will always cause congestion to happen, especially when practically everyone and their grandmother dogpile a new feature. Managing that congestion is not very easy to do, especially when the number of requests potentially far exceeds the capacity that the servers can handle. This is basically akin to the community doing a DDoS attack.

It's also entirely possible to do heavy stress testing in a development environment but then have something unexpected manifest when it's released to the live environment. You try to plan for us much as you can and have a plan for rolling back if necessary if something unexpected happens.

[Kaurhz]

It's also entirely possible to do heavy stress testing in a development environment but then have something unexpected manifest when it's released to the live environment. You try to plan for us much as you can and have a plan for rolling back if necessary if something unexpected happens.

Indeed, the thing that people seem to forget is that simulations are often done in accordance with abstract models and these often don't align very well with the outcomes you find when doing it in a real, live environment, so whilst you may be able to simulate the load, you won't necessarily catch the issues that you might otherwise find when doing it in a live environment.

I sympathize with people being unable to log in or being caught on a DC as a visitor when they may otherwise be set for raiding or have plans with friends. But honestly let's face it, there was a fairly good chance this feature was going to crash in some way or another.

To those that were complaining about the Monday release schedule, this is why.

[NekoMataMata]

And people say that SE is a great dev why? I mean shouldn't something like this have been pretty easy to predict and plan for? This is a pretty big blunder.

[Ransu]

And people say that SE is a great dev why? I mean shouldn't something like this have been pretty easy to predict and plan for? This is a pretty big blunder.

I doubt anyone could predict 15,000 requests per minute....Honestly that's just mind boggling and I can't really understand why so many people want to hop data centers so badly like that outside of maybe the OCE players.

[Stormpeaks]

I doubt anyone could predict 15,000 requests per minute....Honestly that's just mind boggling and I can't really understand why so many people want to hop data centers so badly like that outside of maybe the OCE players.

OCE players cant DC travel.

And yes, this was more than predictable that this would happen, especially considering NA has Balmung and a lot of people would want to go there.