Two things first:
1. There's a lot of QoL suggestions for it out there, and they're good, but that's not what this post about.
2. Despite the criticisms I do think this is a step in the right direction. But it needs work.

So then... there are those that are disappointed or unhappy with the glamour dresser, and then there are those that are flat out afraid of it. Why is this? As it turns out... if you're not careful, you have quite a bit to lose with the new system.

1. The glamour dresser destroys your items. Stage 1 is not so bad, because it's to put it in the new system and you probably will never want to use the gear for stats again anyways. Stage 2, on the other hand -- where you fill up the dresser, is terrible. You're at 200/200, and you have no way to get your old items back. To continue using the system, something has to be completely and irrevocably destroyed.
2. The glamour dresser destroys your dyes. Certain classes of items cannot be added directly to the glamour dresser's inventory, but must be added to the armoire. You have no choice about this. This process destructive to an item's customization.

Of course no one cares about 200 gil dye, but for 300k+ dye it hurts. Nevermind that some people might have bought rare dyes from the mogstation. If they put it on this class of item and want to use said item with new system, they have no choice but to destroy it.

3. The glamour dresser destroys your old glamour. The most minor one, but significant. If you thought you'd use the plate for one job, and let it fall back to the old system for another job sharing gear -- nope. The moment you apply a plate, your old glamour is gone.

tl;dr: I think the biggest folly of the glamour dresser, and one reason it's causing such concern, is how destructive this new system is.

If you read up on it, you can avoid it somewhat (I'm being extremely cautious about what I put in there myself). But for someone that just jumps into using it, they could be in for some nasty, and potentially expensive, surprises.