Update:
New video drivers did not help. I still lag in the city, even when there aren't tons of people on screen. In beta phase 3 I had no issues. They only started appear in phase 4 and beyond.

Perhaps there is a specific graphical setting that could cause my system this laggy trouble? Any one have any suggestions?