Inventory and retainers do indeed overflow if one keeps everything and hoards it "just in case".
If one doesn't do that and instead just keeps the stuff you really need or expect to need within the next few days, then inventory issues practically disappear.

I remember reading some time ago that SE's answer when some people asked for more retainers was that most players do not have any more than the two retainers we all get for free, so they were not planning on letting us hire more retainers.
Or in other words: Most players do not have serious inventory problem.

Only a few, those who are much into both crafting and glamours (or just hoarders on general principles) have serious problems with inventory space.
Personally I do belong to this group and would love to have more space, but on the other hand I realise could probably throw away half of what I have stored and not really miss it so the problem might not lie with the game as such.
(I always tend to end up with full inventories in all other games as well....)