I only have two and could honestly get by with one if I had to. If something is easy to get like bait, dyes, materia, or crafting materials, I just go get what I need when I need it. I mainly use retainers to store things that are a pain to get like potsherds, totems, relics, etc. I do agree there should be a better solution for the relic weapons.