Glamour and why it's poorly designed

[Rongway]

From Thread: Suggestion: Improved Glamour System

I don't know exactly how the current glamour data is stored, but here are my best guesses as someone with a graduate degree in computer science.

There are almost 20,000 equipable items in the game. If you ignore belts, this number will go down; but let's assume the number of items will stay about the same or go up once we account for the items that are coming in Endwalker. That means we need log_2(20,000) --between 15 and 16-- bits to differentiate between all the different equipable items in the game, and another bit for HQ/NQ. So the glamour dresser needs 2 bytes per item to remember what the item is. The glamour dresser also remembers an item's dye. We have something like 120 dyes, and another byte can handle that. There is other information for items in general, but the dresser doesn't need to care about those things, and you're even warned that some data will be lost, such as condition and signature, when you put things in the dresser. So ignoring that information, we need 3 bytes per item in the dresser. A 400 item dresser then should require 1,200 bytes.

If instead we stored 1 bit for each equipable item we've ever obtained, the catalog would require 20,000/8 bytes. That comes out to 2,500 bytes, which is over twice the minimum data required to support a 400 item dresser. So if they're worried about storage space now, it could be much worse with a glamour catalog. There are of course data structure optimizations and compression algorithms that you could apply to reduce the storage required in either case, at the cost of increased processing requirements, but then we get back to how much processing would need to be done on many glamour catalog access requests over a period of time.

I know, 2.5 kB is not a lot. But with 24 million registered players, the total data required to store just glamour data would be about 60 GB.

And that's all before we start talking about how an increase in glamour data size would affect the data transfer volume (and cost), even with it restricted to dressers in inns.

One bit per item, and it wouldn't matter if it's dyable or not because a catalog system would only care if you've acquired the item or not. It would be up to the client to map the bits to items and tell you which ones are dyable.

At 1 bit per item, it comes out to twice as much data as currently required.




I suspect it wouldn't make a difference if the people with multiple characters were allowed to consolidate their glamour data into a single catalog. There are probably far more people with single characters than with many alts, so account-wide glamour catalogs would only save so much space.

Besides, have you ever heard the XIV devs respond to requests for account-wide things? We may some day get a glamour catalog but I don't imagine for a second that they would make it account-wide, even if that could save them enough storage to allow a more accessible catalog.

Original thread date was pre-EW. Following the release of EW:

Honest question: now with the announcement that in 6.2 they are increasing the glamour dresser from 400 to 800 items, I wonder how the memory size difference works out now? Does the current system, doubled in size, still equate to less memory space than a true/false glamour log? I wonder what size of glamour dresser they will have to get to in order to break even, when a glamour log will be smaller than the current implementation of the glamour dresser?

Sure, here are the updated numbers for that.

As of 6.08, there are

4,500 arms
1,024 tools
10,740 armors
3,768 accessories
====
20,032 visible equipment items

IDing 20,032 equipment items requires cieling(log_2(20,032)) bits, or 15 bits. Adding one more bit for HQ brings this up to 16 bits or 2 bytes. Add one more byte for dye information. So the simplest data structure for an 800 slot glamour dresser at 3 bytes per slot would take up 2.4 kB.

Keeping track of a glamour catalog would require 1 bit per item. With 20,032 items, this comes out to 20,032 bits or 2.504 kB.

With 35.8 million subscribers, assuming the vast majority of them only have one character, the total glamour storage difference between an 800 slot dresser and a full catalog might only be about 3.5 GB total among all regions. Note that there are a few items that are exact duplicates (both model and dye channel), so all of these numbers in reality should be slightly less than these calculations, but also I can't say what percentage of the subscriber base has alts, so they could also be higher.

For now, at least, it looks like it wouldn't require storing much more character data if we switched to a glamour catalog. But it's worth noting that these requirements do not scale the same as equipment is added to the game. For a glamour dresser, we could add another 12,500 items to the game and it wouldn't require any additional storage because the minimal size of ID data is the same at 20k items as it is at 32k items. However, adding 12,500 items to a glamour catalog would increase the catalog size to 4.096 kB per character, a linear increase.

Further, a glamour dresser need not take up the maximal space. A dresser that's only half full could take up only half the data storage because empty slots might not need to be explicitly allocated in the data structure. A dresser that's completely empty might take up no storage. However, a catalog would always take up the whole catalog size regardless of how many items you've collected because it still tracks everything you don't have.

So on the one hand, in the present it doesn't look like a catalog would require much more resource than an 800 slot dresser; but on the other hand, maybe they think a dresser will age better?