...For the "job-specific equipment" (i.e. you're being particular to AF/Relic/Empyrean mostly, with only a few other unique pieces to jobs) there's a thing called a Porter Moogle that only takes a few minutes to put gear into and pull it out of. =x

That being said, having 22 inventories of wardrobe will STILL create 22 inventories of server lag regardless of having them "disappear" or "nonexistent" when you switch jobs. The reason is that the data doesn't disappear; it's associated with your character, because otherwise how would they pull the knowledge of what you have in your "vanished" inventory when you switch back? There has to be flags in place that says "they have X gear," and then there has to be information on what those flags mean, which means more processes for each piece of gear they're calling back. I.e. it's really nothing different from the porter moogle; though items don't exist in your inventory, their data flags exist on the item you have in your inventory. Doing so by linking "wardrobes" to yourself will just increase server load times exponentially.