Can we have an update on the situation.

[Starcade]

/clap. A very informed and informative response, thank you. Clearly by what you have mentioned here, just the act of "copying" the servers to another set of servers in another location would also be a rather large time sink.

Speculating, that "could" be what SE is doing. But how do we know? I'll tell you why we don't know and it's because SE IS NOT UPDATING US WITH THE PROGRESS OF GETTING THE SERVERS BACK UP.

BECAUSE THEY CAN'T!!!

Dear God, are people this idiotic???

We don't know right now if half the country of Japan (including Tokyo, I might add) is going to end up effectively glowing green with radiation. The Japanese are consistently getting lied to or tap-danced around, and may I say it does not appear as if they are pleased.

We've got three dead reactors with very live fuel rods exposed to the air, as of the last report I read.

And you're worried about what Square-Enix can do to get the servers back up? You (and many more like you in the last six days) are so insensitive that it is worth it to be inflammatory.

[Ruvion]

Had a feeling this would happen since I've been following the struggle of trying to cool the reactors. Wonder what they're really doing since they won't be able to keep pumping water for the next month. Also wonder what they're doing with all that now radioactive sea water too. SE, since you gave us one free month for a week of downtime, does this mean we are going to get another free month? I've seen people being nice and supportive about the situation in Japan, but seriously how long are these understanding people going to remain, well, understanding. SE has an obligation to support the situation in Japan but also has an obligation to keep it's customers happy. Without any customers there will be no ffxi/ffxiv and are people really willing to wait a month+ or so for a game to come back online. I know they just announced it being down for another week, but seriously, does anyone foresee these problems being fixed that fast? I really think it's going to be at least 1 month before they can focus on trying to restore life to normal in NE Japan. By then it will be mid/late spring and time to start family vacations and enjoying outdoor weather and activities. People that were once consumed by this game will relearn what RL offers and not be as willing to come back, or just find another game to play. Just my 2 cents.

[Ter]

For all those speculating about SE dropping the ball by not having a recovery plan in place I would like to add some clarification. The logistics of replicating data on a realtime system internationally are unbelievably complex. Even if the infrastructure of the country could support the traffic the financial cost of securing guarenteed international bandwidth and redundent routing would be financially prohibitive. The fees paid by FFXI subscribers would not even scratch the surface of these costs and that is why this sort of solution is only really available to high level financial infrastructure. Even if FFXI subscribers paid 10 times as much as they do now this solution would be out of reach.

Even for the biggest companies spending 100s of millions of dollars on their disaster recovery plans would be brought down by a national infrastructure outage. Some disasters are just to widespread to be able to manage effectively. This situation is out of SEs hands and will not be resolved until there is consistant power supply both to the servers and the infrastructure connecting them to the internet.

For those hoping that the data may be moved to a secondary server house in another country I would say it is unlikely. Although there are 24 worlds each of these is made up of many physical servers precisely configured together to form a game environment. I would not be surprised if they were running over 200 physical servers in total (amount really varies depending on the specification of the hardware). Setting up a mirror of this environment is a job that takes weeks at best, even if you can source the hardware quickly. Deploying the software, configuring it and testing the result takes even longer. Migrating the data across on top of this and you have a process that takes a long time. Even if SE did choose to do this they would be facing more downtime when things returned to normal as they would want to move the operation back again, easy physical access is essential when you have so much hardware to maintain.

The japanese people have a lot of serious issues to deal with right now but rest assured one of the top priorities will be getting power distribution back to normal as it is directly damaging the economy. In the meantime we should reflect on how easy the world can be spun into chaos and appreaciate how lucky and priviledged our lives usually are.

Wishing the Japanese populous a speedy and dignified recovery.

[Tummie]

Here we go again. Someone who is crying that those of us complaining that SE isn't more of a responsible company that we are "heartless", "uncompassionate", "addicts", yada yada yada.

I've said time and time again that I feel for the people of Japan and that is genuine. However, i'm not a sympathist by nature. The world has had major disasters everywhere. Life goes on. Business goes on. In fact it should. By grinding everything to a halt (like SE has done in a way by shutting the servers down) it only makes things worse.

I am going to stop you right there and comment. Tokyo Electric Power Company (TEPCO) asked businesses to conserve energy in the crisis. They were conducting rolling blackouts throughout Japan and as many have stated, it is the winter there and it's very cold. Many homes don't have power so SE as a business are following what TEPCO had asked. I am not saying that SE requires a tremendous amount of energy to run their operations, but if it came down to not running the servers so that the extra power that isn't being used can be used by people of need such as families who need power, then it's a no brainier on what to do.

The best thing to do in the face of peril is stand, be strong, don't let the situation beat you, you beat it. It's what drives mankind. If we all threw up our hands and quit things the moment things went sour, then we wouldn't survive. My compassion for the situation of our servers being down has lasted until now. It has changed because SE CAN take steps to bring us back online, and has not. And it also angers some of the population of this game that steps could have been taken BEFORE THIS TRAGEDY EVEN HAPPENED that would have ensured that a game WE PAY for stayed online.

To be honest, I get what you are saying, but we don't know what was going on at SE headquarters. It could have been that they assessed the situation and looked at their backup generators to see if it was feasible to run off of those, but probably felt it wasn't worth the risk should they go without power for long periods of time during the rolling blackouts, and the generators ran out of power. A lot of people would be more pissed off if they kept the game up and all of a sudden without warning the servers ran out of power and everyone R0'd in the middle of something important, and then not knowing when the servers come back up. Things that come to mind are entering dynamis and then having the servers run out of power just as you start. Sure they could do a roll back, but then that makes any progress made non-existent.

Getting really tired of people saying that people that feel the same way I do are told that we are inhuman, callous, hopeless FFXI addicts. I love the game sure but in this off-time I have been doing many things outside of the game, things that I have been doing EVEN BEFORE the game was down. FFXI is not my life, but it's a part of it, I miss it, and i'm angered that it's down when it doesn't have to be. If that makes me a bad person in your eyes, so be it. I'm not on this earth to please you.

Again, it doesn't have to be down, but it's down at the request of TEPCO. Everyone has been raising their voices over having SE relocate the servers to the US, which isn't a very cost effective solution. This whole thing could last a whole 10 days and it wouldn't make sense for them to move their servers to the US, and then move them back.

Also I never said you were a bad person, you just misinterpreted the fact that I said some people act as if FFXI is a drug and they cannot fathom what is going on in Japan and want FFXi to online without really thinking about what is more important to SE.

[Chronofantasy]

Hey everyone,

Most of you may have read the follow-up on PlayOnline news regarding the servers. (for those of you that haven’t here is a link: http://www.playonline.com/ff11us/pol...ews20547.shtml)

First of all I just wanted to say from myself and on behalf of the rest of the Community Team, thank you so much for being patient and understanding in these tragic times. We’re really impressed and moved by how strongly you all support Japan and your efforts to donate.

With the servers being down, many of you are wondering what’s going on with Vana’diel time and your plants, traverser stones, besieged, etc. Pretty much, just think of this as really long server maintenance, which means that unfortunately your plants will be withered, but the number of traverser stones Joachim sets aside for you will have increased, and besiege will pick up where it was left off.

We will give you updates on the situation and again thank you very much for your kind words and compassion.

March 22nd... lol that's the day Dissidia 2 comes out in North America. Well at least they're only extending it for a few days for now. Maybe the game actually will come back quicker than we thought rather than in one month.

Also, people still plant? Is planting still profitable/worth it?

[Tummie]

For all those speculating about SE dropping the ball by not having a recovery plan in place I would like to add some clarification. The logistics of replicating data on a realtime system internationally are unbelievably complex. Even if the infrastructure of the country could support the traffic the financial cost of securing guarenteed international bandwidth and redundent routing would be financially prohibitive. The fees paid by FFXI subscribers would not even scratch the surface of these costs and that is why this sort of solution is only really available to high level financial infrastructure. Even if FFXI subscribers paid 10 times as much as they do now this solution would be out of reach.

Even for the biggest companies spending 100s of millions of dollars on their disaster recovery plans would be brought down by a national infrastructure outage. Some disasters are just to widespread to be able to manage effectively. This situation is out of SEs hands and will not be resolved until there is consistant power supply both to the servers and the infrastructure connecting them to the internet.

For those hoping that the data may be moved to a secondary server house in another country I would say it is unlikely. Although there are 24 worlds each of these is made up of many physical servers precisely configured together to form a game environment. I would not be surprised if they were running over 200 physical servers in total (amount really varies depending on the specification of the hardware). Setting up a mirror of this environment is a job that takes weeks at best, even if you can source the hardware quickly. Deploying the software, configuring it and testing the result takes even longer. Migrating the data across on top of this and you have a process that takes a long time. Even if SE did choose to do this they would be facing more downtime when things returned to normal as they would want to move the operation back again, easy physical access is essential when you have so much hardware to maintain.

The japanese people have a lot of serious issues to deal with right now but rest assured one of the top priorities will be getting power distribution back to normal as it is directly damaging the economy. In the meantime we should reflect on how easy the world can be spun into chaos and appreaciate how lucky and priviledged our lives usually are.

Wishing the Japanese populous a speedy and dignified recovery.

This ^ and I thank you for explaining exactly what I was going to. Moving the servers isn't as simple as moving a workstation from one to another. THey'd have to re-configure hardware, test, make sure they have enough bandwidth to support their application, all of which takes a lot of time. Moving a data center (Which SE has done in the past) takes half a day, but we are talking about SE replicating a full environment and data.

[Ter]

No problem Tummie. This is what I do for a living so I can appreciate how difficult the situation is. I thought a more realistic and accurate appraisal might help some of the people here understand the scale of the problem.

[AJ_Anyia]

For all those speculating about SE dropping the ball by not having a recovery plan in place I would like to add some clarification. The logistics of replicating data on a realtime system internationally are unbelievably complex. Even if the infrastructure of the country could support the traffic the financial cost of securing guarenteed international bandwidth and redundent routing would be financially prohibitive. The fees paid by FFXI subscribers would not even scratch the surface of these costs and that is why this sort of solution is only really available to high level financial infrastructure. Even if FFXI subscribers paid 10 times as much as they do now this solution would be out of reach.

Even for the biggest companies spending 100s of millions of dollars on their disaster recovery plans would be brought down by a national infrastructure outage. Some disasters are just to widespread to be able to manage effectively. This situation is out of SEs hands and will not be resolved until there is consistant power supply both to the servers and the infrastructure connecting them to the internet.

For those hoping that the data may be moved to a secondary server house in another country I would say it is unlikely. Although there are 24 worlds each of these is made up of many physical servers precisely configured together to form a game environment. I would not be surprised if they were running over 200 physical servers in total (amount really varies depending on the specification of the hardware). Setting up a mirror of this environment is a job that takes weeks at best, even if you can source the hardware quickly. Deploying the software, configuring it and testing the result takes even longer. Migrating the data across on top of this and you have a process that takes a long time. Even if SE did choose to do this they would be facing more downtime when things returned to normal as they would want to move the operation back again, easy physical access is essential when you have so much hardware to maintain.

The japanese people have a lot of serious issues to deal with right now but rest assured one of the top priorities will be getting power distribution back to normal as it is directly damaging the economy. In the meantime we should reflect on how easy the world can be spun into chaos and appreaciate how lucky and priviledged our lives usually are.

Wishing the Japanese populous a speedy and dignified recovery.

Yeah, what he said. lol

[Jrtaz]

Hey everyone,

Most of you may have read the follow-up on PlayOnline news regarding the servers. (for those of you that haven’t here is a link: http://www.playonline.com/ff11us/pol...ews20547.shtml)

First of all I just wanted to say from myself and on behalf of the rest of the Community Team, thank you so much for being patient and understanding in these tragic times. We’re really impressed and moved by how strongly you all support Japan and your efforts to donate.

With the servers being down, many of you are wondering what’s going on with Vana’diel time and your plants, traverser stones, besieged, etc. Pretty much, just think of this as really long server maintenance, which means that unfortunately your plants will be withered, but the number of traverser stones Joachim sets aside for you will have increased, and besiege will pick up where it was left off.

We will give you updates on the situation and again thank you very much for your kind words and compassion.

Sweet its great to hear, i always thought, it freeze everything inside thanks for the update SE

[Silvermane]

I think everyone understands and agrees with your analysis of the complex nature of any kind of relocation project. However I also think people are suprised, maybe even disappointed that SE had no contingency plan in place other than turning off the servers to preserve game data.

We also know that they do have game data backup and recovery systems in place, so why not have a "shadow" server network where the backup data was backed up again. Then all that would be required would be the activation of this network.

I reiterate its the lack of foresight which I think conserns people espically when you consider the fact that Japan is not stranger to such events and other than the cost benefits of centralisation why have this equipment in Japan at all?