That's just turning old game limitations into sacred cows.

If there is ever a cinematic with NPC's riding them around in town your idea that they are lore-wise not allowed in towns flies out the window.