That's a problem with fishing. While Botanists and Miners can pretty much pick what they want to gather, Fishermen cast their line and hope. I tend to keep all Seafood, even where I keep only High Quality Botanist/Miner items, just because the fish can be harder to gather again when I want them.
(CUL is right, but Alchemist is generally abbreviated ALC.)
Like other items, crafting materials specify an item type in their tooltip description. As a Culinarian, you're mainly going to want Ingredients, plus sometimes Seafood. As an Alchemist, you'll mainly want Reagents and sometimes Ingredients. Keep those three categories, particularly the HQ ones. Leather, Bone, Stone, Metal, Wood, and Cloth are really only going to come into play if you decide to level the other crafts as well.
That said, you may well end up deciding to level all of them. Just as with battle classes, crafting classes can use cross-class skills, but unlike jobs that can only pull in skills from a couple other classes, crafting classes can pull in skills from all seven of the other crafting classes, and some of those cross-class skills become pretty important for being able to produce High Quality results. There are a set of cross-class-able skills (one from each class) at level 15 that include some important ones. Another set at level 37 you can pretty much ignore, because the only important one from that group comes from Culinarian, which you'll be getting anyway since that class is one of your primary focuses. You have to get all the way to level 50 to reach the next important cross-class skills.
The low level materials that you'll need in getting to level 15 can be bought from vendors if you don't have them on hand from your own gathering. (The vendors in the crafting guild halls have a more complete selection available for what their particular class needs than the more general vendors in the city's merchant area.) So even if you sell off all your Leather, Bone, Stone, Metal, Wood, and Cloth, you'll still be able to level the other crafting classes far enough to reach that first set of cross-class skills just by buying the needed materials at the fixed and relatively low prices that vendors charge. I'd aim for that level 15 point for the other 6 classes while taking CUL and ALC the rest of the way up. Once you've finished leveling those two, you can decide whether its worth going back to any or all of the others and taking them far enough to get their high level cross class skills. Don't worry about their materials until then.
I eventually gave in and and got a third retainer, despite having the extra fee involved. My extra retainer holds excess gear and some miscellaneous items, so that my two initial (free) retainers' inventory can be devoted exclusively to crafting materials. That works so far, though it's tight at times. I'm working on all 8 DoH classes, but haven't finished leveling yet. (Level 50 Weaver, 40 Carpenter and Goldsmith, 38 Culinarian and Blacksmith, 37 Leatherworker, 31 Alchemist, and 30 Armorer.)
If you decide to only take your two primary DoH classes past level 15, so aren't keeping all the categories of materials, then you can probably get by with just the two free retainers. If you decide to take all eight of them all the way up to where they give you those level 50 cross-class skills, then an extra purchased retainer or two would make it a lot easier.
That's more true for some classes than others, though. Culinarian and Alchemist are probably the most self-sufficient in terms of materials. They'll occasionally use materials from each other, but very rarely need anything from the other six.
In some cases you need both stuff from your current level plus stuff from previous levels as well, though. As a level 50 Weaver, for instance, I'm still using cotton fairly often, which I've been using since around level 10. Alchemist still keeps using level 1 quicksilver all the way into Heavensward recipes in the mid 50's.