It can be a problem. Just doing MSQ and flying quests from 50 to 60 is going to net you a hundred items and that doesn't include green items from dungeons.

I took it to the extreme but I made 8 characters on Sarg, leveled them all to 30 or so and unlocked their retainers. In addition I bought 2 extra slots so I have 32 retainers. I could have another 32 if I wanted to spend the extra money but for people who want to arrange things they find, doing it by level and using alts is the way to go. The truth is, you won't ever use most of it and should get rid of it but that system will allow you to figure that out on your own.