Servers in Danger of Crashing RIGHT NOW!

Quote:

---

Originally Posted by Zarzak

uhh... 25 slots in armory... 24 classes. You are right because no one should ever carry around that 1 extra weapon -_-

---

Man, I know some people that have /multiple gear sets/ for the same job to accomodate different accuracy caps in order to push higher DPS on earlier-turned fights back in FCOB. Taht right there is devotion. xD

I dislike the retainer system all together. Having to select one, wait for it to come, click through the dialog only to find out it's not the retainer you stored the item you want in is kinda annoying. One of those "bottom of the list" annoyances, but an annoyance nonetheless.

The guild chest is a better system IMO. We should be able to pay our GCs to store and sell items from personal chests. Wouldn't that take up less data than retainers? Retainers have gear, personalized character data, dialog and animations. Chests are just chests. And chests are MUCH EASIER to navigate through...

At some point I hope they stop adding retainers and give us a personal GC chest option. I'm not asking them to take away retainers - just, as I said, give us an option. I realize we have a chest in the inn, but it's limited for some reason. I'm no game designer so I could be wrong about chests taking up less data... but I don't see why they wouldn't.

Quote:

---

Originally Posted by Phenidate

I'm a software engineer.

I can tell you right now that though we speak different languages, a japanese byte is the same size as an american byte is the same size as a zimbabwe byte.

edit: also I should be in there as Aika Niceboat, 1.0 player ;)

And yet they have enough space set aside for us to buy how many retainers again?

---

Not going to lie, but your comments really impressed me. :P {Please keep commenting as you do.}

Quote:

---

Originally Posted by Phenidate

Thats for legacy accounts. Don't count the "40 characters max" because thats spread across multiple worlds. Multiple worlds wouldn't be running on the same hardware.

And thats Legacy players. I'm going to guess that there's a lot more standard accounts than legacy at the moment.

I was in the midst of changing my post since max total across characters doesn't make sense to the argument given that you could only be logged into one character at a time.

---

Actually no, its not just legacy, the standard sub for 14.99 a month also gives you 8 chars per server, and 40 total. Only the entry level sub gives you one character per server, up to 8 I believe.

and you counted 11 characters in Wow, which you can't be logged into at once. So...

I am a bit annoyed that retainers are the "answer" to inventory problems. I keep different sets for mining to avoid repair costs... etc.

Quote:

---

Originally Posted by Gamer3427

In addition to obvious arguments about how it's not about data storage, but transfer, you're also not accounting for the many tiny variations between one item and the next. Items that are stored retain quite a few flags such as whether it's HQ vs NQ on any non gear items

---

Come on now. Even someone with mere basic knowledge such as myself will tell you that a half decent IT engineer would simply make NQ and HQ different item ID's. Why needlessly complicate things with ''flags''?
The only flags I know of are:
-durability
-spiritbond ratio
-collectibility
-creator
-materia


Btw the top 3 of those are client side only (the data is shared between only you and the server) , which is evident by them not showing when you link the item. Glamours are not counted as its simply 2 different IDs being used.
Dyes are the same, different ID for each color.


Displaying all of these as flags is a really inefficient way of doing things. The only reason for doing it like that would be if your computing power far exceeded your storage capacity. Which is obviously not the case here.

Quote:

---

Originally Posted by Gamer3427

Also, I don't know what games you've been playing, but compared to most MMORPGs and even most single player offline RPGs, we have quite a lot of inventory space, as pointed out in the post I'm quoting below.....

---

Most MMORPGs? Single player RPGs? oO
Okay how about WoW? Vastly more space then we have here. And on top of that even more space via the use of alts, your mailbox, and a guildbank.

Yet despite not having 7 billion reagents for crafting, or the need to have 10 different sets of equipment (''only'' 4 or 5 at most); they still increased storage by adding a reagent tab and increasing stack sizes to 200.

FFXIV has one of the worst inventories (compared to the total amount of items) I've seen anywhere. Offline or online.

Quote:

---

Originally Posted by Archulak

If you're going to count alts as inventory space then try doing it for ffxiv:

8 chars x (450 inventory slots + 300 armory slots) = possible 6000 slots per account per server.

math is fun aint it.
(also before the obvious "It's not convenient to move items around between alts in this game", that dosn't mean that it can't be done)

---

Barely anyone has alts in this game. Whereas in a game like wow you can expect people to have several alts. (long time players even 8~ on a single server)

Please do not use alts as an argument for storage space, because alts arent really supported here. You cant even mail an alt.

Quote:

---

Originally Posted by Eidolon

For all intents and purposes, our characters have more than just the 100 slots offered by the bag. We have the Armory Chest, as well.
100 Inventory Slots + 12x25 = 300. So 400 inventory slots on our character, where games like WoW actually shares the inventory for armor/items

---

Yeah im sure the collection of 40 staves I had on my mage in WoW would fit real nice in our armoury system. And im sure the 25 slots for boots would be real great to fit in my pair of 2 boots and 23 items that arent boots.


Shared inventory also means no slot is a wasted slot.

Quote:

---

Originally Posted by Zabuza

Yes, because WoW puts ALL of their current focus into their raids, neglecting everything else. FF spreads out their content resources over many aspects of endgame, not just raids for shinies.

---

Like crafting and gathering, yeah?

Quote:

---

Originally Posted by Phenidate

Yeaaah shoot I forgot the armory space.

But even still, with caching and hashing they should be able to accommodate it without stretching to the limits of what their hardware is capable of handling.

Note though, that they make you open the armoury chest seperately - so then is that a separate call to load inventory from the server? It doesn't open simultaneously with your bag after all.

---

Armoury space being taken up is a design choice anyway. How many of these slots would become empty if their class/job requirements were abolished?
On top of that, I often have thrash in many of my armoury spaces because why not. Its not like I can place a stack of materials in the pants section of the armoury? xD

XIV's inventory system is worlds better than XI's was. Due to a memory limitation on PS2, the most any given bag could hold was 80 items. So we ended up with Gobbie Bags, Mog Satchels, Mog Sacks, and all sorts of variations, all holding 80 items max each.

The only slot I really have a problem with in the armory right now is my rings - if anything I want them to double the armory storage of the ring slot, since each class has TWO rings and I've been getting the "can't change classes" warnings a lot lately because I upgrade a ring and forget to pull out the old one.

Quote:

---

Originally Posted by Aeyis

Barely anyone has alts in this game. Whereas in a game like wow you can expect people to have several alts. (long time players even 8~ on a single server)

Please do not use alts as an argument for storage space, because alts arent really supported here. You cant even mail an alt.

---

I know quite a few people that have about 3 - 5 alts actually, and just because "People don't do it" isn't really an excuse to discount the fact that they could.

(Also I'm the one that said already its inconvenient to transfer items between alts so it's funny to me that you try and use it as a counter argument)

Quote:

---

Originally Posted by Raist

Each one of those 24 slots is another query to the backend databases---that is a big increase in traffic if you math it out to 400k+ users in real time.

---

I am shocked to hear actual engineering concepts on an online forum, +1 and I am impressed.

Very well put, I was about to type the same.

Quote:

---

Originally Posted by Archulak

I know quite a few people that have about 3 - 5 alts actually, and just because "People don't do it" isn't really an excuse to discount the fact that they could.

---

I know absolutely noone that does. In fact many people I know were surprised to learn that I had an alt at all! Just one alt was a major surprise!

''People dont do it'', is not an excuse. Its a base for logic. Why base things on the assumption that people use alts, when in reality they dont?

Quote:

---

Originally Posted by Archulak

(Also I'm the one that said already its inconvenient to transfer items between alts so it's funny to me that you try and use it as a counter argument)

---

Perhaps I misread then, because the way I read it you said something along the lines of (or well rather, exactly like)
:

Quote:

---

Originally Posted by Archulak

(also before the obvious "It's not convenient to move items around between alts in this game", that dosn't mean that it can't be done)

---

Which means pretty much the opposite..

Quote:

---

Originally Posted by Aeyis

Which means pretty much the opposite..

---

It means that someone is going to be super wise and say "Its inconvenient to move items across alts therefore you can completely discount it". To which my counterpoint in that same sentence is you can't say that no one will do it because I know people who do and it must be given consideration.

So yes you did miss understand hope that clears that up.

Also if you're building a system you don't say "Oh this elevator can carry about 500 lbs but i don't have to have safeguards in place, in case someone goes over that, cause most people won't", you need to account for possibilities that exceed your expectation or you can come across massive failures and at that point your only option is to say "Hmm didn't think anyone would do that."

I guess most of you don't understand how databases work.

The point of their statement was about transmitting the data. And yes, it isn't 400k users on one world, but we knew over a year ago that we had crossed the 5000 simultaneous login threshold per world. We also know that they increased capacity past 7000 shortly after that, with a goal to eventually reach as high as 15,000. That is simultaneous connections per world.

Even just mathing it out at the point of 7000 capacity, 24 slots at your simplified example of 32bits per ID would increase the query count potential to an additional 168,000 queries--and requiring north of 5Mbit/second bandwidth to support it (7000*32*24=5,376,000). That is per world and it doesn't account for any handshaking, overhead, etc...that is just the expected data stream for transferring the raw data in one direction. Keep in mind also that it has to flow both ways (client to server, and server to client), so it could easily become 11Mbits combined just for one world to move only the raw item ID back and forth.

Now, add in the extra information that may be bound to the ID as well (materia, Spirit Bond and Repair levels, etc.)---it wouldn't be too far a stretch to see the potential to have a spike in traffic that could consume up to the entire bandwidth of a single 10 Gigabit card across the entire playerbase. Remember, we have over 1million subs--have to prepare for the possibility that concurrent connections may ramp up at a moment's notice. Just using the simple math of 32 bits and 400k users---that is north of 3Gbits in one direction, more than 6Gbits in full duplex at the current active player levels.

There are currently 60 some-odd worlds, split up in two continents...NA has just over 30 worlds. Soon there will be another data center in Europe as well. There is much potential and expectation for future growth in play here. Keep in mind also that this isn't just internal data transfers across the backend either--all this traffic is also relayed to the clients over the internet as well.

Now consider the real-life conundrum that is at the heart of North America's internet latency/stability woes. There is a known, documented issue with overselling last mile bandwidth against the established upstream bandwidth at the exchange points to transit/peering partners throughout North America. Multiple filings with the FCC by some top tier ISP's (specifically TATA and later Level3) going back as far as 2011 complained/warned about increased internet consumption in the US and how the last mile ISP's were neglecting to upgrade their upstream bandwidth capacity at the exchanges, and the potential for it to bottleneck the internet in general. Couple that with filings over the next few years trying to get them to operate under fair practices (eventually led to exposing the Comcast forced throttling of Netflix traffic). Testimonies were cited where top tier ISP's offered to share the cost to install 10 Gigabit ports, even offering to install and wire everything up at their expense...and those last mile ISP's STILL refused to upgrade.

Just this past year, the same accounts were cited online and before the FCC and Congress (Net Neutrality, Time Warner mergers, Comcast/Verizon scuffles...etc.). MAJOR INTERNET PROVIDERS were publicly shamed for over-utilized, overly congested, and flat out failing upstream capacity--and the simple fixes being proposed were to install additional blocks of 10Gbit bandwidth. In some cases, just even ONE card would have provided 25% more bandwidth at exchange points that were pegging no less than 90% utilization and would have been a HUGE improvement in staving off the congestive failures. In that one specific example, there was space for up to 8 cards, and only 4 were installed. And still... those ISP's refused to install just one single 10 Gigabit card, and the bottleneck continued to spiral towards congestive failure.

So...we have an established problem where the ISP's in North America want to dicker for nearly 5 years over installing 10Gigabit cards to support the increased bandwidth they have been selling to us....and you expect there to be no impact whatsoever if SE makes a "simple" change to provide 24 more slots that could have the potential to throw 10Gigabit/second worth of traffic on the internet....yes it is not per world, but combined it will have an impact. Remember, there are 30 some-odd worlds hosted in Canada at the moment, and we have seen documentation where 10Gbit upgrades are needed just to properly support the existing traffic flow we currently have.

There is far more to this problem than just having enough hard drive space to store the data.

Quote:

---

Originally Posted by Phenidate

Maybe ask them what's going to happen when that returning players campaign kicks off and thousands of inventories SLAM the servers?

Maybe ask how they plan to handle the game getting any new players, adding 450 massive inventory slots each.

Apparently we just can't spare the space!

---

It's useful to think before you post. Let's look at a full server like Balmung. Now, if they suddenly increased the data for EVERYONE on the server, that could potentially cause server problems. But if it's only a handful of people buying extra retainers, that's not an issue.

You can't create new characters on Balmung for the most part, so they don't need to worry about new players or those coming back.

On low pop servers, increasing inventory wouldn't be an issue. It's the high pop servers they have to worry about. I'm not sure how many people there are per server, but let's say 3,000 for argument's sake. You give 3,000 another 24 slots and that's 48,000 new slots. To equal that, people would have to buy 500 Retainers.

Also, maybe Yoshi-P wasn't thinking just 24 slots per person when he said the servers might have issues, maybe he was thinking another Retainer's worth. And then the gap gets that much bigger.

ItemID 24 bits
GlamourID 24 bits
Color: 8 bits (256 color options, conservative but high - unless they do R,G,B,A,M (M would be a bit, metallic, 1 or 0)
Durability: 15 bits (it's decimal, hidden, 8 bits for 0-200 + 7 bits for comfortable decimal up to 1/127th)
Spiritbond: 15 bits (as above)
Creator: 32 bits (it'd be an ID, not their name, however for transfer the name would be sent - say 1024 bits)
Materia: 120 bits (at most 5 item ID's)

238 bits for storage (< 30 bytes)

95,200 bits for a character inventory (11,900 bytes - 11.62kb) that might be openable at any time so is the worst case scenario.

11.62kb x 15,000 players = 170mb written to disk every 15 seconds for inventory


Position/Direction: 128 bytes
Animation: 1 byte
Selected Target: 4 bytes
Selected Target Type: 1/2 byte (player, NPC, object etc - point of reference for facial expressions)
Character ID: 4 bytes
Character Name: 128 bytes
Character Visible Gear: 27 bytes
Job: 3/4 byte (56 possibilities)

TOTAL: 293.25 bytes

64 players on screen: 18,768 bytes - and some people get lag at this point showing their bandwidth is saturated.
When it comes to storing characters however, you only need the ID and the position/direction.

Characters are there for reference to show you how much data they store about things that are not inventory stuff.

tl;dr: Inventory is a tiny part of the data they store and send out to clients - so small in fact that even at capacity of a server (if everyone was online at once) any modern SSD's would store the data far quicker than they currently write it out.

Quote:

---

Originally Posted by Phenidate

WOW: The biggest bag you can have is 30 slots. They support 136 slots of inventory with the backpack.
Open the bank? 210 slots max.

---

On WoW:

the reagent bank: 98 slots where all items that can go into there stack to 200 a piece
Void Storage: another 80 items (Equipment only)
Guild storage: 98 slots per tab, 8 tabs total, equaling another 784 slots

and your personal bank starts with 28 slots, which can be expanded by 30x7 currently, which totals 238

that gives you 1200 (I didn't count for on-hand inventory either, only storage that's held off-hand) on the dot for storage, and I'm not even counting alts at all. Also you can craft from your bank, you don't need the items on hand. There's not a lot that WoW does great, but inventory is superbly handled imo.

Just turn the augmented AF into armoury already. I had much gear to put on retainer's inventory right now :( (And stuff.)

How would you guys suggest they fix the issue with data???? How can they improve the data center? Can a data center be upgraded???

I did some quick maths about inventory space once. If you increased inventory stack limits by one byte for every slot to allow bigger stacks for 8 million characters (4 million registered accounts and two characters per account), it would increase the amount of storage needed by just over 7GB. Increasing stack sizes from 100 to 200 would not even require any extra space at all, unless Squenix is using signed integers for items for god knows what reason.

Inventory space is nothing compared to everything else they have to store and send about our characters.

Quote:

---

Originally Posted by Phenidate

So compared to other MMOs, either our servers are crap (which is ridiculous when we have so many players paying monthly mind you) or our inventory is so horribly innefficient that even flavor of the month korean F2P games do it better than FFXIV.

Let me break out some mumbo jumbo to put into context the amount of data we're talking about in an ideal inventory:
Every item in your bag is just an item ID and a quantity, that's about all the server needs to know and tell your client. It knows what item it is by looking up the id. It knows how much you have by the quantity.
All the items info, graphics etc are stored client side. Using the item ID it looks up the graphics, descriptions, etc from the data on your computer.

We know that items are stored client side since sites like XIVDB regularly datamine it from the client.
So thats all an inventory stored in a database should be - the item id of each item in each slot, and a number signifying it's quantity.

Do you want to know how big an item ID and a number would be? They can both be numbers, and if we're going to be extravagant they're full on integers. 32bits each. So each slot would take up two integers.
How big is an integer? 32 bits.
How many slots do we have in our bag? 100. 175 per retainer.
2 numbers for each slot, that's 64 bits.
100 slots that's 6400 bits.
Each retainer by the way would be 11200.
Bear with me now: there's 8 bits in a byte. Our bag is 800 bytes. Our retainer is 1400 bytes.
Let me put that in terms of a quantity you may know: kilobytes. A KB is 1000 bytes, so our bag is .8KB, Each retainer is 1.4KB.
Our bags plus two retainers is 3.6 kilobytes.

To put that into perspective, a 1TB hard drive holds around 1,000,000,000 kilobytes. So that's what, 50 bucks to accommodate 277,777,777.778 players. I don't think we have 278 million players right now.

Want to know something else? I used waaaaaaaaaaay too much data for the quantity, since the most you can have is 99 of any item. We can just use a byte (8 bits) for the quantity. The item id being an integer by the way, gives us enough space to identify 4,294,967,296 unique items. We could make the item ID even smaller to accommodate say, 2 billionish instead.

Simply put, you could fit the inventory into an even smaller number of bits than what I gave above if you were to optimize it.

"Hey Phenidate, whats all this mean to me?"

Well it means that if the people who engineered the inventory system were trying to be good engineers and not waste space, the amount of data of everything you own could fit onto a grain of rice.

---

It's not about the storage space, it's about the transfer. I agree that it is something that can be addressed but it was already stated by a dev that it is a problem with how often the server is queried to save our inventory. It saves ALL of our active inventory somewhere like every 15 seconds. For every character on every server. Retainers are not included in that save because they are not active at all times. They are saved when an inventory change is requested.

The answer they gave is very simplified in order to get the point across to a large number of players that don't know anything about servers, databases, etc.
While some of you know more about those topics than the larger playerbase, their answer lacks a lot of critical information that would be necessary to actually make a point about it all.

In other words, don't read too much into what was being said.

Quote:

---

Originally Posted by Phenidate

Thats right, if everyone were to buy a retainer, the servers could come crashing down on us right now according to YoshiP.

---

Charcter inventory, that the character have always access and can change every moment =/= retainer inventory, that have to summon first and always only 1 retainer at the same time.

I'd rather have stable servers though than Squenix messing with them just so someone can have an extra few inventory slots to store those tier 1 materias in. You do have to be careful comparing it to WoW though considering they have had almost ten years now to deal with their stuff, Squenix has not had that long. They will get there, it just won't be right this second so best manage your inventories better.

Yet at the same time, we're free to purchase multiple extra retainers.

Quote:

---

Originally Posted by Nekodar

The answer they gave is very simplified in order to get the point across to a large number of players that don't know anything about servers, databases, etc.
While some of you know more about those topics than the larger playerbase, their answer lacks a lot of critical information that would be necessary to actually make a point about it all.

In other words, don't read too much into what was being said.

---

Actually the responce was quite specific and you don't have to "read anything into it". Pretty straight forward what their concerns are. I've bolded the "critical information" for you:

Quote:

---

since additional inventory space would increase the data transmitted, there is a risk for server crashes, so this isn’t something we can do right away

---

Quote:

---

Originally Posted by Raist

Actually the responce was quite specific and you don't have to "read anything into it". Pretty straight forward what their concerns are. I've bolded the "critical information" for you:

---

I can read just fine, thank you. And like I said, it is a very simplified answer they provided that is lacking a lot of information.

Quote:

---

Originally Posted by Nekodar

I can read just fine, thank you. And like I said, it is a very simplified answer they provided that is lacking a lot of information.

---

Ummm.. It isn't rocket science. Forums are littered with complaints about lag, login queues, disconnects, slow downloads/loading times, etc. The response clearly stated the increase would increase the amount of data to transmit and that might lead to more instability.

There should be no need for them to provide further clarification/explanation. Stability and response times are already an issue with the current internet/network loads. Making a change that would instantly increase that load by a flat number (with numbers suggested would be like 14% across the board on our inventory side) would further exacerbate a known problem.

Does the retainer system seem a little counter-intuitive to anyone? On one hand, it's a little innovative and novel to have a sort of NPC servant handle things for your. On the other, it is a little clunky. Or something. I can't describe it really, it just feels...weird.

I am no IT wizard by any means, but, let me get this straight. If problem is server/database storage, then giving me an extra retainer will add to the stress and crash the servers, but my paying for 1 or 2 or 6 would not cause this problem?

If transfer rate is the problem, then giving me 1 extra retainer would cause too much load and cause server crashes, but if I pay for up to 6 more it will not? Maybe someone smarter than me can explain, because right now I am still scratching my head over this one.

As others have said, if they really truly can't figure out how to expand our armoury chest space or personal inventory, they definitely can give us all another free retainer as compensation.

Quote:

---

Originally Posted by SarcasmMisser

You know this whole thing could be worked around effortlessly if they allowed us to send items to alts and they could act as mules, all diablo style.

---

Pretty much this. I think it's time for SE to start seriously looking into having an account-wide bank like a lot of other MMOs do. I know Aion had a system like this. You place the item in a chest and another character on your account can pick it up.

Quote:

---

Originally Posted by Kaaylryn

I am no IT wizard by any means, but, let me get this straight. If problem is server/database storage, then giving me an extra retainer will add to the stress and crash the servers, but my paying for 1 or 2 or 6 would not cause this problem?

---

It's simple.

Give people six free retainers = EVERYONE would make six retainers.
Give people the option to pay for six extra retainers = only a limited number of people will do it.

It's a way to allow those people who feel that they absolutely must have all that inventory space to have it while discouraging the majority of people from hoarding gear and items.

Quote:

---

Originally Posted by LunaHoshino

Give people six free retainers = EVERYONE would make six retainers.
Give people the option to pay for six extra retainers = only a limited number of people will do it.

---

Actually, it's more like:

Give everyone one free retainer = everyone would use one additional retainer.
Give people the option to pay for six extra retainers = just as many if not more retainers would be created and used as above, but only by people who are willing to overlook an obvious moneygrab.

I'm still waiting for the third "free" (because we're already paying a sub that should take care of these QoL features) retainer that was mentioned some time ago. Meanwhile, we've got an expansion with new crafting mats, new gear sets, three new jobs (more gear slots needed), 10 new levels (even more gear slots needed, and more than what we needed from 1-49, as now each set has a job restriction and you can't just use the same to level multiple jobs), new tokens (Alexander, for instance, with different tokens from both Normal and Savage), etc. All that without a single extra inventory slot. Just a quick question: Are we expected to sell/destroy/whatever the gear of our level 1-59 classes/jobs, or the glamourable sets (yeah, imagine crafting the thavnairian sets just to be "forced" to delete them in the future), or the "youcan'tstorethiseventiteminthearmoireyet" items, and buy/craft/farm them again when we need/want to use them? Or is that our fault too, because of poor inventory management yadda yadda?

I seriously doubt that adding another retainer would stress the servers that much, especially when those are not updated all at once every 15 seconds. It would only require more server space (a tiny bit), but it wouldn't affect data transfers.

This is going to get worse with each subsequent patch, once more and more items are added. And paid retainers just add insult to injury, as our inventory issues should have been resolved before doing anything like that.

Here here!