My armory is a disaster as well. Mostly because dungeon drops and leve rewards default to the armory and it's stuff I will never use but never take the time to clear out. For crafting if you level them all together the one set will cover them all. For gathering as well, though I keep my Gathering gear as I level because I use it to level retainers.