I was chalking this up as lag due to characters all congregating in the same place BUT it doesn't happen when I'm in the market wards at all, even a really full one.
I've messed with the settings and eventually resolved to put up with it or just stop running to where ever I am headed for a moment until it "calms down".