I have a GTX680, 32gb of ram, and a high end i7 cpu and just like you I have bad performance but only in certain places.
While I think the game could be somewhat better optimized, I think that maybe the overpopulation is exacerbating the performance drops in places like cities since in my experience with the game the big frame rate drops tend to be when I get near areas where people are gathered.