With the current retainer interface, just two is way too much work.
They need a single menu, with instant switching between them at any level. No dialogs, quit from any level, craft from retainer inventory, drag/drop materials between retainers and bags.