I'm the same as you: outfits that come out of the fantasy world I have in my head tend to disturb me. I hate soubrettes and other wacky ‘disguises’ (frogs, I hate you).
But then... In practical terms, in a game like FFXIV where you can dress up in modern clothes (leather jackets, jeans, converse) and other... Well, there's not much point in limiting glamour any more. If visual consistency is important to you, you're already obliged to ignore a good proportion of the glamours you come across IG. In that sense, allowing people to expand their mirage boards won't really have any impact.