The symbol means it is indeed a piece of gear saved onto a Gear Set and getting rid of it will result in that gear set having a "missing piece". You will get a notice when a Gear Set is either missing a piece or has a change to it's meld or glamor, different from what was originally saved, when you activate that Gear Set.
If you have gear in your Armory Chest that doesn't have that symbol, then you can get rid of that piece without effecting any saved Gear Sets.

To add, if a gear item has a Blue X in the lower corners, it means it is an item that is not equipable by your current job, but you do have a job that can equip it.
An item with a Red X means you do not have a job that can use the piece.