I have that problem as well. They just take up space and can't be discarded. There was an idea I was going to do.

Have you tried storing your AFs in the retainers and then releasing that retainer? By releasing the retainer, I assume whatever they have on them would be lost. Of course, before you do this, make sure to move the items out you still need.

I'm haven't tried yet since the retainers keeping all my crafting mats.