Server Issues and Limitations

[Sorel]

Now tell me would you understand if SE said "we apologise for the bad latency issues, we have has a consultant in and have been advised that we should change the priorities on all out routers because they arent load balancing enough, Also they have suggested that we reconfigure all routers to use OSPF.

We were told that spanning tree was not functioning correctly which alos impieded the performance of out network.

That is just a small amount jargon that would be in a letter they would tell us. if you know what OSPF, Spanning tree and the specific reasons priorities matter and how routers perform load balancing then its ok, but this kind of letter would go into that much technical detail only a few players will understand because you have to be educated in routing and switching to know what there talking about.

And it's not like we are asking for detailed explanations, IP addresses, or blocks of code here. A sentence or two with nouns other than "server" and "issue" would be enough. But they're not even giving us that.

Use your words, SE. We're not kindergarten students or silver-haired grandmothers. If you happen to use a word that we don't understand, we'll Wikipedia it and get the idea.

Again ... we not asking for every explicit detail of the problem. Just some further explanation other than "server issues".

It's almost like we're back in the old Hiromichi Tanaka days.

Interviewer: Many players have complaints about <<fill in the blank>>. What are your plans to address them?

Tanaka: We cannot say at this time, but please look forward to it!!

Now with Naoki Yoshida at the helm, communication about fixes to current issues and future plans has become much much better, EXCEPT when it comes to the subject of "fixing server issues" ... which BTW is preventing half a dozen other game issues from being resolved.

Seriously, what's up with that??

[Delsus]

Again ... we not asking for every explicit detail of the problem. Just some further explanation other than "server issues".

It's almost like we're back in the old Hiromichi Tanaka days.

Interviewer: Many players have complaints about <<fill in the blank>>. What are your plans to address them?

Tanaka: We cannot say at this time, but please look forward to it!!

Now with Naoki Yoshida at the helm, communication about fixes to current issues and future plans has become much much better, EXCEPT when it comes to the subject of "fixing server issues" ... which BTW is preventing half a dozen other game issues from being resolved.

Seriously, what's up with that??

The server and network problems are probably technical and if they were going to give us inforemation about it they would have to go into detail rather than get a completly ambiguous comment like "we have noticed problems with the router/server/switch/firewall configurations and are working to correct them" if I got that (being educated in networking) I would know that it could take anywhere from a few weeks (if they knew the exact problem and has everything completly documented) to a few months and I would need to know more, a router config for a large company could have hundreds of commands that need to be sifted through to fix somthing.

Sometimes it is alot easier to wipe a configuration and start from scratch, but that can bring the network down if it isn't done carefully, and takes awhile to get clearance which is why companies rarely do that, they take the time sifting through the commands.

Again all they can say is "we know there are server/network problems and we are working to address them" or give us details about the problems even the smallest detail about a network can open it up to hacks, who needs IP configs if they give us one protocol they use I can use nslookup to get the ip of one of thier servers then launch a ping sweep of thier whole network through that open port from one protocol they leaked.

If you think they can give us more info without giving us info then give me an example that isnt just "we know there are problems and we are working to resolve them"

[Sorel]

If you think they can give us more info without giving us info then give me an example that isnt just "we know there are problems and we are working to resolve them"

Possible Examples:

We are working to reconfigure server storage to allow for <<insert feature here>>, but it will take some time.

We are working to steamline network communications to improve <<insert feature here>>.

We are planning on updating our databases to make it easier to <<insert feature here>>, but we won't be able to complete that until after <<insert feature here>> is implemented.

And the Development Team doesn't have to explain every little detail when they do this. They don't when they talk to us about other aspects of the game. They don't detail the new crafting algorithms to us when they tell us about crafting ... they don't tell us the additional lighting and shader effects when they talk about improved animations ... and they don't tell us the polygon count when they announce new armor.

So I don't see why the Development Team would be forced to give us every detail of their internal IT structure when they talk to us about the "server issues" that are plaguing FFXIV.

We just want the Development Team to communicate about the server issues the same way they communicate about fixing every other issue in the game.

[Delsus]

Possible Examples:







And the Development Team doesn't have to explain every little detail when they do this. They don't when they talk to us about other aspects of the game. They don't detail the new crafting algorithms to us when they tell us about crafting ... they don't tell us the additional lighting and shader effects when they talk about improved animations ... and they don't tell us the polygon count when they announce new armor.

So I don't see why the Development Team would be forced to give us every detail of their internal IT structure when they talk to us about the "server issues" that are plaguing FFXIV.

We just want the Development Team to communicate about the server issues the same way they communicate about fixing every other issue in the game.

There still as ambiguous as "server issues" they dont tell me anything except they want to add new features, all those would do is fuel the arguments that SE dont know what the problems are, I would prefer them to say nothing, then see in patch notes that the server and network infrastructure have been optimised.

If I heard those I would want to know the problems because I know that streamlining network comminications can take a few seconds if it is an interface priority problem preventing effective loadballancing, or a few weeks if they have to order new interfaces and configure said interfaces, lets also not forget that other part of the configuration could need changing for an interface replacement.

And if they need to replace outdated routers and switches, you better hope they have thier configurations backed up...

[VydarrTyr]

Again ... we not asking for every explicit detail of the problem. Just some further explanation other than "server issues".

Well, you're asking for that now. But the OP isn't asking for it. And to be honest, I can't remember anyone else ever asking for it. So while I would agree with those suggesting that it would be a bad idea for SE to reveal that info, at the very least we can hopefully show some patience and wait for an explanation until after we've asked.

As for the OP, here's a bit of the Famitsu interview posted by the incredible, inimitable, subligar-sporting Reinhart:

Famitsu: How about delivery?

Yoshida: We will implement delivery, but it will be after server modifications. Message and letter will be same.

Famitsu: When a player has something for repair up in bazaar and has item full you cannot repair that persons damaged equipment. Are there any plans to fix this?

Yoshida: It will be hard to do right away but we are going to make changes how the trade window works so along with that we would like to work on this.

...

Famitsu: Do you have plans for changes to party search feature?

Yoshida: First we will make minimum required changes to the current UI and with server repair planning to release redesigned search feature. In patch 1.18 part of it will be modified. Forming parties with search feature means forming with random people and it might troublesome for some (laugh) so some just try it once and get angry and don’t use again…. It’s right that easier search feature and UI is needed but lastly it’s really up to the player to have courage.

Famitsu: How about the area search feature?

Yoshida: Along with party search this is also going to be redesigned

Famitsu: Will it be possible to show more details for linkshell members?

Yoshida: Not only linkshell but for friendlist same type of system is required so it is being looked into carefully.

Famitsu: What kind of measures is being taken for inventory space as with armory system you will require to carry a lot of equipment?

Yoshida: might say not that again, but this will be after repairs to the root. Plan is already made but will need to be changed along with swapping the UI. Will try to make it by PS3 release but currently can’t give exact date on it yet.

Hope that resolves your issue.

[Sorel]

Well, you're asking for that now. But the OP isn't asking for it. And to be honest, I can't remember anyone else ever asking for it. So while I would agree with those suggesting that it would be a bad idea for SE to reveal that info, at the very least we can hopefully show some patience and wait for an explanation until after we've asked.

The subject of the "server issues" and "server optimizations" and "server repairs" has been one of the recurring themes of several threads, but there have been only two threads that I could find that directly asks the Development Team about these "server issues". This thread is one, and the other thread is below.

http://forum.square-enix.com/ffxiv/t...lly-addressed.

Now granted, both threads are less than a week old. I'll give you that. So I think the call for patience in waiting for a response to these server issue questions is not unfounded.

Nonetheless, question has now been officially asked, the challenge has been formally issued, and the gauntlet is now irrevocably thrown.

Now we wait, and see what the Development Team has to say on this.

[captainpicard]

Perhaps SE folks did the art work, and outsourced the rest of it to a North Korean or Chinese developer. If you have ever worked in project management, with complex vendor relationships, its very easy for the vendor to start dropping the ball, or leave you completely screwed. Perhaps the vendor fulfilled the contractual obligations with SE, and the result was the game we received at launch. Perhaps the vendor deferred or deleted any number of requirements from the project due to failure to deliver, and so what we got was half a product that barely works. Now the SE staff has scrambled to actually try and 1) understand the vendor code 2) begin to try and modify it to improve upon things.

It may sound far fetched, but we are dealing with this very scenario on one of the projects at my employer.

It would explain a lot. If it were true, it explains a whole lot about why the game launched like it did. i.e. - we poured X dollars into the vendor relationship, at minimum lets recoup what we can on boxed sales.

Meanwhile, stubbornly, rather than just kill it, they have endeavored to keep it on life support, with the far fetched hope that it may be profitable one day, IF they can get it into a better place with as few resources as possible. If they had a real solid number of people working on this project, it would be much further along, and thats a fact. I think they are throwing as few resources at it as possible in the hopes that they can push it out again on PS3 (boxed sales - more revenue recoupment) charge a monthly fee to a small audience that is just enough to cover the costs that went into getting it to that state, and cross their fingers that due to HOW BAD IT IS, people will say HOW FAR ITS COME and be blind to the reality that this sucker plays like its 1999.

That said I don't hate the game, but its in a woefully precarious place.

[Delsus]

Perhaps SE folks did the art work, and outsourced the rest of it to a North Korean or Chinese developer. If you have ever worked in project management, with complex vendor relationships, its very easy for the vendor to start dropping the ball, or leave you completely screwed. Perhaps the vendor fulfilled the contractual obligations with SE, and the result was the game we received at launch. Perhaps the vendor deferred or deleted any number of requirements from the project due to failure to deliver, and so what we got was half a product that barely works. Now the SE staff has scrambled to actually try and 1) understand the vendor code 2) begin to try and modify it to improve upon things.

It may sound far fetched, but we are dealing with this very scenario on one of the projects at my employer.

It would explain a lot. If it were true, it explains a whole lot about why the game launched like it did. i.e. - we poured X dollars into the vendor relationship, at minimum lets recoup what we can on boxed sales.

Meanwhile, stubbornly, rather than just kill it, they have endeavored to keep it on life support, with the far fetched hope that it may be profitable one day, IF they can get it into a better place with as few resources as possible. If they had a real solid number of people working on this project, it would be much further along, and thats a fact. I think they are throwing as few resources at it as possible in the hopes that they can push it out again on PS3 (boxed sales - more revenue recoupment) charge a monthly fee to a small audience that is just enough to cover the costs that went into getting it to that state, and cross their fingers that due to HOW BAD IT IS, people will say HOW FAR ITS COME and be blind to the reality that this sucker plays like its 1999.

That said I don't hate the game, but its in a woefully precarious place.

SE would never outsource a MMO that will be constantly updated for its life, which they want to be as long as possible, if they outsource the server builds, the servers are not SE's property and cant touch them, only buy new ones, there are plenty of companies with servers that just arent plugged in because they cant touch them and the vendors did not collect them after relationships broke down, there are also servers that are running and the company doesnt want to switch them off because they could be important.

Also with the software builds outsourcing them is just plain stupid for a games manufacturer, you know people who make games.

It is impossible to speculate what goes on behind the scenes because the possibilities are endless. Yes there are programming problems and network/server problems but these can be hard to find without starting from scratch, and poor programming makes it alot harder, if the programming is really bad it can take a team of 5 people twice as long as one person using good code to do one thing.

The problem is not the amount of staff its the quality of the origanal progamming which makes everyones life harder. If they want to work on the core programming they will need to pull people from other projects to keep pushing content and features out, so they will probably do that a bit later because it is needed.

[captainpicard]

SE would never outsource a MMO that will be constantly updated for its life, which they want to be as long as possible, if they outsource the server builds, the servers are not SE's property and cant touch them, only buy new ones, there are plenty of companies with servers that just arent plugged in because they cant touch them and the vendors did not collect them after relationships broke down, there are also servers that are running and the company doesnt want to switch them off because they could be important.

Also with the software builds outsourcing them is just plain stupid for a games manufacturer, you know people who make games.

It is impossible to speculate what goes on behind the scenes because the possibilities are endless. Yes there are programming problems and network/server problems but these can be hard to find without starting from scratch, and poor programming makes it alot harder, if the programming is really bad it can take a team of 5 people twice as long as one person using good code to do one thing.

The problem is not the amount of staff its the quality of the origanal progamming which makes everyones life harder. If they want to work on the core programming they will need to pull people from other projects to keep pushing content and features out, so they will probably do that a bit later because it is needed.

I think you make a lot of valid points here and it fairly kills my idea that they outsourced the code.

I just can't get past the actual changes they have made in the game. When I look at whats really been changed, it seems like very small pieces of code. Lets evaluate a few examples with the understanding that in my opinion, these are not major coding feats that should take the kind of time we are talking about:

1. Adding NM to the various zones = add a mob with a hidden level that drops a new item and has a unique name
2. Change skill point calculation from random to predictable = delete randomizer, tie player level to mob level, create base sp modifer, add group bonus
3. Simplify recipes = add new simple recipes with same outcome as more complex recipes, delete complex reciples from db later
4. Make hotbar not disappear = "hotbar.disappear = false;"
5. Change hotbar graphics = draw new icons, replace old icons with new ones
6. Speed up (if you call it that) retainer interactions = delete line of code that was requesting server confirmation every time the user moused over an item
7. Events = cut and paste, new skins, new text
8. Delete Stamina Bar = /* comment stamina bar code */ add timers to each skill

You see where I am going with this. The change we have seen over the past year since release, when evaluated honestly, is not at least from the outsider looking in standpoint, major code changes. If this was your C# program, then you know how easy these things would be to accomplish in very little time.

The fact that simple things like kick user from linkshell, or mounts, or airships, and so forth have not been added as of the date of this posting tells me either

1. There is next to no staff working on this project
2. The staff working on this project doesn't understand the code
3. The staff working on this project are very, very bad coders
4. All of the above

Take your pick. I mean really there is no explainable reason for the lack of overall sweeping change. That is what led me to speculate on this games development having been partly outsourced.

When you look at the quality of some of SE's none MMO titles, it looks as if a completely different company developed them.

I also seem to recall an announcement around the time of launch that there would be Chinese servers and the game would be served up by a Chinese company, and modified to suit that audience appropriately, which also makes me wonder if said company didn't actually author the game itself or have a hand in some of the code we see in effect today.

We have gotten incremental change. You typically give incremental change when you don't want to inadvertently take down critical systems. Baby steps. They have already stated that player retention is not a goal at this time, and they are not charging for the game, so this is the time to make those changes that could take down the system. What do they have to lose?

I am just stumped.

[Delsus]

I think you make a lot of valid points here and it fairly kills my idea that they outsourced the code.

I just can't get past the actual changes they have made in the game. When I look at whats really been changed, it seems like very small pieces of code. Lets evaluate a few examples with the understanding that in my opinion, these are not major coding feats that should take the kind of time we are talking about:

1. Adding NM to the various zones = add a mob with a hidden level that drops a new item and has a unique name
2. Change skill point calculation from random to predictable = delete randomizer, tie player level to mob level, create base sp modifer, add group bonus
3. Simplify recipes = add new simple recipes with same outcome as more complex recipes, delete complex reciples from db later
4. Make hotbar not disappear = "hotbar.disappear = false;"
5. Change hotbar graphics = draw new icons, replace old icons with new ones
6. Speed up (if you call it that) retainer interactions = delete line of code that was requesting server confirmation every time the user moused over an item
7. Events = cut and paste, new skins, new text
8. Delete Stamina Bar = /* comment stamina bar code */ add timers to each skill

You see where I am going with this. The change we have seen over the past year since release, when evaluated honestly, is not at least from the outsider looking in standpoint, major code changes. If this was your C# program, then you know how easy these things would be to accomplish in very little time.

The fact that simple things like kick user from linkshell, or mounts, or airships, and so forth have not been added as of the date of this posting tells me either

1. There is next to no staff working on this project
2. The staff working on this project doesn't understand the code
3. The staff working on this project are very, very bad coders
4. All of the above

Take your pick. I mean really there is no explainable reason for the lack of overall sweeping change. That is what led me to speculate on this games development having been partly outsourced.

When you look at the quality of some of SE's none MMO titles, it looks as if a completely different company developed them.

I also seem to recall an announcement around the time of launch that there would be Chinese servers and the game would be served up by a Chinese company, and modified to suit that audience appropriately, which also makes me wonder if said company didn't actually author the game itself or have a hand in some of the code we see in effect today.

We have gotten incremental change. You typically give incremental change when you don't want to inadvertently take down critical systems. Baby steps. They have already stated that player retention is not a goal at this time, and they are not charging for the game, so this is the time to make those changes that could take down the system. What do they have to lose?

I am just stumped.

They were doing those small things because people wanted content, and this was the fastest way to give it to us, and for NMs the only thing that is the same as other mobs of the same type is the look, everything else is differant, darkhold and totorak are not copy and paste they are completly new areas, you make it all sound like you could do it in a day, but if you are a programmer you will know about changes having unforseen problems later on like the recent ffxi cure bug.

For people that dont know that was when cures would do 99,999 to any undead mob regardless of resistance when cure was not touched in the update, this is why they spend a long time testing updates to ensure as few of these game breaking bugs go into the game.

You also failed to mention the animation changes for auto attack, which meant taking a trip to the motion capture room.

All the changes take time because they need to plan, code, test, recode, test etc before they go into the game.