If there is lag for some people, it's likely on their (SE's) network side the problem is. Be it directly on SE's network, SE's ISP's network or the upstream providers.
I can only say that I have not seen any sign of such lag myself, which indicates that it depends on which path the traffic takes to the datacenter.