Long term idea. Mini wards.

[solracht]

Gee, how will we ever get around that issue? Maybe by restricting retainers to certain areas?

I assume you're planning on not letting them be placed in areas such as: adventurer's guild, aetherytes, around the repair npcs, around the retainer bells, around the support npcs in the guilds, around the guild counters, city exits?

What's the point then? Attaining some kind of fake sense of immersion by having hundreds of zombies standing in whichever hallway is the most popular, and probably ruining that part of the city for a lot of people? Because I think you're delusional if you think the result from your idea would be a nice, immersive city with player vendors all over the place.

You'd get the wards outside the wards. At least the retainers don't bother anyone there and we won't even have to step in if they adjust the system as they should.

The one or two random guys who might set up a shop by the faraway camps will be cool to have, have to admit that. That could be an entirely different idea altogether however (such as the one actually suggested by the OP), but there's no need to take steps back to implement that.