Yup. It was a monumentally poor design decision in 1.0 that we've been stuck with ever since. Crafting actually locks your state while you're doing it. So, no retainer access, no accepting invites, no rearranging items in your inventory, no nothing. That's all due to the initial mistake of not using an event model for state.
Technical debt like that is hard to fix after so long. Important, but hard. I'm not sure they have the will to actually do it at this point given how many things it could affect and how it could caused reduced stability initially (even if it'd pay off massively in the long run).
Or they'll just put crafting itself into the phone app, which doesn't carry the same technical debt and would likely be easier. Plus then they can charge $2/crafted item for it.It should be a simple enough matter to load and edit retainer inventory during crafting. But given what SE has done lately, I'd expect there to be "technical issues" with doing it that magically disappear if you pay more for your subscription.