This is actually a feature implemented to encourage players to wear, er, minimalist clothing. It didn't take me long after switching to Au Ra to realize that the endgame glam meta is to just not wear upper body clothing at all.

So let down your hair and let loose your inner naturist.