That's not how it works. I work in a datacenter. Compression is impossible, a file compressed can be used unless the data your compressing needs to be referenced or updated(each time it did would require an un-compress then re-compress meaning you need to save space for the files expansion). so compression is a no go. plus the additional IO and transfer of data to read all this data that updates if you move any icon, switch any item and so on would be bad practice. for example. websites cache stuff locally to a browser to stop the front end from having to constantly query the backend database.
so to list a few flaws in storing the data on a server in a data center.
-additional monthly cost from data center to "store" data.
-additional IO that would require additional monthly fees to provide from data center.
-new design to support the amount of database queries to all clients simultaneously
#Data centers charge by the GB of space. getting 10PB would likely cost more per month then most of you make in a year.
Data center storage is not cheap... this is not paying 30 dollars for a 1TB drive. Server storage is on a whole different level and is not cheap.
Not saying its not possible. just this is done by design and not later. it could call for recoded server structure. the easiest way would be to limit what is saved. Like TERA who keep a small config file that gets pulled down locally and updated when modified and sent to server. though what they save is very limited.
Last edited by Noata; 02-14-2014 at 11:33 PM.
Not saying that you are wrong - quite the opposite - but there are a few ways to mitigate the storage space dilemma. They could store a diff file, for instance. If I were to make a gross guess, very few users actually make many changes to their macros or have many gear sets. As such, if you store only the users' alterations, rather than the individual data itself, you could drastically reduce the overhead. Depending on the format of the diff file, you could quickly and efficiently load macro and setting changes without permuting TBs of data. That's a pretty superficial improvement though.
On a a more abstract level, you could use a flyweight to enumerate commonalities between gear sets. If two people have the same gear set, you can use 16 bytes instead of 28, if 3 people use the same gear set, that goes to 17:42. Hotbars are the real sticking point, there's only so many ways to visualize that data. The issue is that hotbars are too general-purpose, abilities would be easy enough to map, but macros, gear sets, and emotes greatly increase the range of data that needs to be available. Best case scenario is the number of hotbar items * 2 bytes + 4 bytes (for region markers so that you can overload bits).
Bottom line, with a few clever data permutations, in the end, you're going to end up with ~6kB (assuming 2 bytes per item) per character for hotbars and 1.7kB (assuming 4 bytes per item) per character for gearsets. Of course, since you're assuming macros are static resources, you'll have to make those available as well, that's another 40 bytes (4 byte command * number of lines) per macro, so plop on another 4kB on the front. So call it a cool 22kB per character.
To use the number (which I am not sure where it comes from, but I'll keep things consistent) stated above, if there are 60 million accounts - active and inactive - you would spend ~1.32TB of space storing the data server side. Please by all means check my math, I do have a bad habit of loosing orders of magnitude form time to time, but I have gone over these numbers quite few times now, and I'm fairly confident in them.
It's worth noting that you could substantially reduce the bandwidth strain as well by only downloading and uploading the file on session creation and session deletion. It would cause occasional funkiness, such as when you crash after writing a new macro, but it would not effect the discussion at hand, which is transferring data to a new device. You could also mitigate the above by making downloads only constructive, while uploads are both constructive and destructive. That way you wouldn't lose new macros on a crash, but they wouldn't appear on new machines until you correctly shut down.
Your math is quite far off the mark, first because you multiplied by the maximum number of characters someone could have and then gave the result as "per character", and secondly because you multiplied the gearset information times the number of hotbar slots.
So instead of: 40 * 30 * 14 * 100 * 60mil = 100.8 TB
it would be: ((30 * 14) + 100) * 60mil = 31.2 GB
Now, as you said, some of that info may have more than one byte, so the real amount could be significantly higher than 31.2 GB, but it wouldn't be anywhere near 100 TB, nor even 1 TB for that matter.
The hotbar slots would each hold the ID number of either a skill, an item, or a macro slot. Even if the ID numbers they use are 64 bit, that's only 8 bytes, and they're more likely to be 32 bits, or 4 bytes each, since that would still provide more than enough unique ID numbers for the number of skills and items likely to ever be in the game.
The gear set slots hold an item ID and a dye color ID. (They don't store the rest of the details about an item. If you have two items of the same type and color, switching to the gear set will take whichever is currently stored first in your armoury chest.) That's only two numbers, probably 4 or at most 8 bytes for the item ID and one byte for the color.
So assuming the most likely 5 bytes per gearset item and 4 bytes per hotbar slot, along with your numbers from earlier we get:
((30 gearsets * 14 items * 5 bytes) + (100 hotbar slots * 4 bytes)) * 60mil characters = 150 GB total
or if the ID numbers are 64 bit, it would be:
((30 gearsets * 14 items * 9 bytes) + (100 hotbar slots * 8 bytes)) * 60mil characters = 274.8 GB total
You're assuming the pieces would be updated individually in real-time. That would be one option, but a far more cost effective one would be to update the entire structure just once at logout.
Last edited by Niwashi; 02-15-2014 at 02:30 AM.
We're way off topic at this point, but I just wanted to point out that you forgot that the new system coming out next update prioritizes by the correct materia, so it need materia info as well, so append another 5 bytes on there. Also, remember that each gear set has its own hotbar sets, so it's not ((30 * 14) + 100) * 60mil = 31.2 GB, but (30 * (14 + 100)) * 60 mil = 205.2 GB.Originally Posted by Niwashi
Ah, I didn't know about that. Nice.
That's partially right (and closer to right than I was in my previous post). Gear sets don't have their own hotbar slots, but classes and jobs do, so with 19 classes and 9 jobs it would be
((30 * 14) + (28 * 100)) * 60mil = 193.2 GB
and with reasonable sizes added (including that extra 5 bytes for materia you mentioned for the next update) it comes to:
((30 gearsets * 14 items * 10 bytes) + (28 classes&jobs * 100 hotbar slots * 4 bytes)) * 60mil characters = 924 GB total
So it could be getting near the 1 TB level if every character ever created has all 19 classes, all 9 jobs, and all possible hotbars enabled and class/job specific. If they only store the parts that are actually in use, though, they'd probably save most of that space due to characters without all the classes and jobs, or who only use a couple hotbars for some of their classes. That pattern just needs a small overhead for marking which pieces are in use and which aren't.
Last edited by Niwashi; 02-15-2014 at 04:10 AM.
dat meteor wipe on king behemoth lol
|
![]() |
![]() |
![]() |
|