Create Overflow Servers Tech Dudes - Not a rage post

[Design]

Yeah that's right, I saw this posted somewhere else in one of the 100+ forum pages that no one is probably reading anymore.

1. So what you should do is, go contact someone at Guild Wars 2, and ask them how they made their overflow servers. THIS solved their log in issues when they first released the game and lowered congestion on the main servers until they fixed all the problems.


2. Increase all servers (don't make 50 new ones just make the ones you have now way bigger). It's better to go too big then not big enough.

Overflow also is what Blizzard did with WoW (cross realm leveling) and GW2 still has today (even though sadly they have nowhere near the need for it anymore).

[Design]

Nooo don't go to page 2...

[Souly]

I just wonder why i don't get into the queue...
I keeped trying to log in, after 5 min i was logged in without a queue, i think the queue is just not working atm.

[FraterOrmus]

Servers shouldn't be 1:1 to begin with (this isn't the 20th century anymore and MMOs shouldn't be built that way in the 21st century). Server Worlds should be Clustered for Load-Balancing, High Availability, and Scaleability. At minimum, they should be 3:1 (three physical or virtual servers per Game World), and then scaled up according to Peak Capacity x 2 (generally you should plan to stress your servers to a max of only 50% capacity when at Peak). When a World begins to Peak past 50% capacity you add more servers to that Cluster until the Peak drops below or at 50% capacity. This is Systems Administration 101.

Some MMOs cap a game world at 4,000-10,000 players, adding new Clusters (Game Worlds) rather than expanding existing Clusters to keep Game Worlds from seeming too overpopulated. That works too as long as your existing Clusters can handle Peak Load x 2.

True Story:

The MMO Dark Age of Camelot (DAoC) was the successor to the once popular text based MUD (Multi User Dungeon, the precursor to MMOs) on GEnie and AOL. Mythic anticipated 10,000 users at Launch, but smartly planned their capacity for 30,000 users (3x expected capacity). Launch went smooth, the champagne was uncorked, at which point the Lead Developer and CIO of Mythic glances at the login numbers and says "Guys, we have a serious problem!" Turns out they had 50,000 people show up right at Launch (and 250,000 shortly thereafter). Even planning for 3x the capacity they Stress Tested for, they were still unable to handle that many players. So, what did they do? First thing, the CEO tried finding somewhere that could overnight them enough Servers on the weekend and bought every computer that was for sale at retail in the city on a Friday night, and while he was doing that the Lead Developer/CIO started making Image Backups of everyone's workstations (they had 25 Devs at the time) and putting those workstations immediately to work as temporary Servers to handle the sudden influx of players until new Servers arrived. It wasn't pretty from an IT stand-point, but it kept things running smoothly until they got 100 new Servers delivered to their Data Center, then installed & configured. During it all, no players noticed a thing. From everyone else's perspective it was a successful launch and DAoC went on to be awarded "Game of the Year" (the success of which caused Microsoft to steal their Source Code to launch their own duplicate MMO called "Mythica", and attracted EA Games to buy out Mythic).

The moral of this story is that success can either make or break your MMO. Having too many customers can be a mixed blessing if you are aren't ready to handle it or aren't quick to handle it in any way you can. I can totally understand Square Enix being surprised at the success of their Open Beta (when all of these problems first began), but they totally dropped the ball when faced with higher numbers of logins than they had planned for. Before Phase 4 of Beta was over they should have been scrambling to scale their Clusters (or if not Clustered to begin with to be Scaleable, they should have Clustered them at that point) in anticipation of Early Access and certainly in time for Official Launch. Considering that these things only came as an after-thought, when limiting capacity and then limiting concurrent logins were their Plan A and Plan B respectively, speaks very poorly about them, and casts serious doubts on whether they are capable of successfully running an MMO.

Especially considering that FFXIV:ARR was supposed to be an apology for the disaster that was FFXIV, you'd think they would have tried harder to not allow it to become as equal of a disaster. I guess some companies never learn. Fool me once Square Enix and shame on you! Fool me twice Square Enix and shame on me!

[Shaone]

Servers shouldn't be 1:1 to begin with (this isn't the 20th century anymore and MMOs shouldn't be built that way in the 21st century). Server Worlds should be Clustered for Load-Balancing, High Availability, and Scaleability. At minimum, they should be 3:1 (three physical or virtual servers per Game World), and then scaled up according to Peak Capacity x 2 (generally you should plan to stress your servers to a max of only 50% capacity when at Peak). When a World begins to Peak past 50% capacity you add more servers to that Cluster until the Peak drops below or at 50% capacity. This is Systems Administration 101.

Some problems you can't just throw more servers at, and that's Software Architecture 101. The performance you can gain from clustering is very highly dependent on the workload involved.

For instance, if you have a computationally heavy workload that can easily be subdivided into logical separate units, then there is pretty much no limit to the gains you can get from clustering. See SETI@home or similar.

However, if you have a workload consisting of lots of small interdependent calculations that cannot easily be subdivided, and each node requires up to the moment data to function, and so each change another node made will need to be propagated to the other nodes. In this scenario you will see ever diminishing returns as you add more nodes, until you see no gains from new nodes, as data channels become flooded keeping those changes synced. In this case you may get more benefit focusing on the processing power per node over node count.

Now, XIV unfortunately mostly deal with the second type of workload, at least within any given locale/zone. So I hope they have some real beast servers on order, and designate whole servers to the busiest zones. Subdividing past zone level is going to be tricky, which is why the OP suggests overflow servers if a specific zone becomes overloaded. The example used (Guild Wars 2) uses overflows if a zone becomes crowded for precisely this reason. Back to this shortly.

If I were to hazard a guess I'd say it's probably in the 10s of servers per world just for zone processing, with each server handling either one busy zone or a number of smaller zones. And then probably a number of edge nodes for the master database. Maybe a few servers for player communication channels (linkshells/whispers etc). Then perhaps a general shared pool of 30 or 40 servers for duty finder instances (I'm sure each one can handle hundreds of simultaneous dungeon runs). These numbers are just wild guesses though, and I have no problem admitting that. I would love SE to be more open with their architecture, I think people would be surprised and impressed with what it really takes.

So, to summarise, processing of zones subdivides nicely, and appears to be the main form of load balancing employed for this game (see service statuses where specific zones in a certain world become unavailable, this is most likely a world node fail).

Especially considering that FFXIV:ARR was supposed to be an apology for the disaster that was FFXIV, you'd think they would have tried harder to not allow it to become as equal of a disaster. I guess some companies never learn. Fool me once Square Enix and shame on you! Fool me twice Square Enix and shame on me!

It's a pity you feel that way. I think the game is amazing and they are clearly in for the long haul, so I am prepared to be a bit more forgiving. Also I don't have any problem logging in when I want to play, guess that's benefit of being in a less popular region (EU)
___________________________________

Now on the subject of the thread.

Overflows in GW2 are cross world. And the game is much less restrictive about cross world trading etc, as shown by the trading post. All worlds are more or less created equal. XIV is not very cross world, and those cases where players are thrown together they are not permitted to trade items or gil, because some servers have a new economy and some have a legacy economy with a hell of a lot more money floating round. Some people don't want the legacy "taint" (not sure I agree, but I can understand).

So I think zone overflows as implemented in GW2 would have to be as heavily restricted as duty finder instances, or be world locked so they only contained players from a single world. Not a problem, just a tradeoff to decide on if they went this route.

Anyway, despite the potential annoyance with teleporting to meet something only to find you are in overflow and can't see/trade with them, I support the idea of overflows as a possible mitigation for overcrowding. Even if the data centre upgrade fixes the issues.

[KraggyKor]

Nooo don't go to page 2...

Why, what value has this thread got?

[Atrious]

Why, what value has this thread got?

What value does your reply have?