SUGGESTION: memory and I/O efficient way to a store glamour catalog

During the E3 2017 Letter to the Producer Live, Yoshi-P implied making a glamour catalog would be hard...

Of course I have no knowledge of the internal representation of object in FFXIV, but I would like to offer the following suggestions to SE's software engineering team:

Use a representation such as a bit array (01010101111100000), if the at the object id index the bit is 1 then the character has the item available for glamour, if the bit is set to 0 then the character doesn't have the item. If you had a million piece of gear that array would be ~123 KByte big without compression. This bit array should be pretty easy to compress to store and transfer. Various optimizations are possible on the array (range exlusions for items that are not glamourable etc.) Lookup in the array is O(1), so it's fast.

You also don't need to send that glamour gear bit array each time the player change zones, you only need to update it when the player login and when they get new gear. To verify if the array changed, a simple checksum can be exchanged rather than the full array. A more complete lookup can be done when the player try to apply a glamour (e.g. to recover the dye color of the item the user want to glamour.)

The same concept can be used for armoire since it doesn't store materias, nor dyes, nor item status (spritbonding/condition).

Quote:

---

Originally Posted by Dartagnan

The same concept can be used for armoire since it doesn't store materias, nor dyes, nor item status (spritbonding/condition).

---

I'm pretty sure the same concept already is used for the armoire, and that that's the reason why it doesn't store any of that extra information. But yes, a glamour log would work the same way.

And if instead of just indexing the object id (since the majority of objects aren't gear), they simply assigned a position in that bit array to each piece of visible gear, it could be a lot smaller than 123 KB. According to Lodestone, (excluding belts and off-hand tools) there are currently 9099 pieces of gear in the game. Of course, it will be more than that by Friday, and will continue to grow thereafter, but even just a couple kilobytes would allow for storing 16384 gear pieces, a roughly 80% increase from the total that exists now.

Storage and memory really shouldn't be an issue for something like this. It would, of course, take some work setting up the user interface for accessing / finding the items you want from it, and their UI team has been pretty busy with all the other features being added in Stormblood, but I really wish the glamour catalog were one of them, or that it's at least the next thing to get added to the game. It would certainly be a lot more helpful than some of the other new things we're getting.

I wonder, does it always have to be data that's loaded on said character? Can it be something that is called upon? I'm no programmer, so it just seems odd to me that this kinda thing can't be called up on request from another server, something like retainers.

Again, have no clue how SE has things work when it comes to our items or how to store a images data once obtained for the first time.

Quote:

---

Originally Posted by Esp

I wonder, does it always have to be data that's loaded on said character? Can it be something that is called upon? I'm no programmer, so it just seems odd to me that this kinda thing can't be called up on request from another server, something like retainers.

---

Depending on how they design it, it could go either way.

Some people's requests for a glamour log included the request that whenever we receive new gear, or at least whenever a piece of gear is bound to us, it's automatically added to the log. That particular version of the feature would require it being kept with the character.

An alternative would be a version where we go somewhere to interact with the feature (just like we interact with a summon bell to access our retainers) and from there we manually add the pieces whose appearances we want it to store and/or use them to glamour other pieces. That version could be just called up when needed and wouldn't need to be kept in memory the rest of the time.

But there's an in-between version, which would actually be my choice if they were to ask my opinion on the matter. It could be useful to have a feature (like summon bells) that we need to interact with in order to add new items to our glamour log, but then let us use that log to glamour other gear from anywhere. Even though the glamour log would need to be kept in memory for that version, it's still a lot less demanding on the system, and that's because of SE's backup process.

There was an interview quite a while back where they explained that one of the biggest difficulties with adding additional inventory space is that every few seconds they copy the character information back to the database for everyone who's logged in in order to ensure that all recent changes are preserved in the case of a server crash. That's a lot of data to back up with only a few seconds in which to do it, so adding to the size of that data can put a strain on the system. But with a glamour log feature, items aren't removed when they're used for glamouring other gear. The only time the contents of the glamour log would change is when new items are added. So if the glamour log is only added to while interacting with a summon-bell-like feature, then that's the only time its data needs to be backed up. We could still be given access to using that data to glamour stuff without it being part of the every few seconds backup process, since the data wouldn't change during that part.