There are a large number of ingredients that aren't heavily used. For example, inks and their ingredients are only used for AST and ACN weapons.

Many materials can be purchased from vendors in cities or (if your FC has a house) at the housing vendor.

Finally, many ingredients are only useful at certain level ranges, most significantly by CUL and ALC, and to a lesser extent by GSM. The easiest way to look at this is the just-updated ffxivcrafting. For example, by setting it to look at every CUL recipe between level 37 and 160, we can check whether a material is used by that craft within that range. If it isn't, chances are pretty good that we'll never use it, and can thus sell it. You'll want to be a little more careful when you reach the level cap -- there's reasons you might want to make Bouilibaise rather than Seafood Stew -- but as you get higher in level you'll have a better idea of what that involves.