You lose the Materia on that armor upon removal anyways, so the deal here is that you have to be careful when deciding upon which materia to add to your armor.