I'd rather have quality then freebies. The money they make off us goes to development. Frankly, the current path FFXIV is on, an upcoming expansion, new dungeons in the most recent patch (with BEAUTIFUL graphics by the way), other improvements such as the upcoming Golden Saucer, choco racing, games, etc . . . all of this is possible due to budget. So, no, leave the retainers where they are.

In the year or so I have been playing this game I have never had an inventory issue. If you pack-rat enough where you need to be featured on the show "Hoarders", then you have the option to purchase extra space to feed your pack-ratting habit.