If anything they should just add an debuff for not wearing the right gear in battle like -15% to the stats the gear gives