It is pay to win, if your goal is to be a market baron. Being able to access your retainers and AH while at work is pretty big. Thankfully that's only a tiny portion of the base, and gil is not needed to progress really.

The frustration is that it never ends here.

Want to water your plants out of game? Moogle coin. Want to play with your house? Moogle coin. Dye your character's gear? Play all your toybox minigames? The potential for add-on services is large, and I think a lot of the anger that is justified comes from this.