You can buy additional retainers.

Though really, with the release of Heavensward they should at least given us a free third retainer. They could double the armory space and I'd be golden. Maybe also give us another 50 slots in our Inventory.