Are you playing in borderless window mode? If so there's no concrete solution assuming you want to continue to play in borderless window mode. A band-aid fix is to swap to full screen mode every now and then after visiting heavily populated areas, alt-tab (so the game gets minimized completely), tab back in, change back to borderless window mode.

Doing that forces the game to dump unused memory which also seems to prevent the game from eventually crashing. The bug has been around since forever, even prior 2.1 and there's still no signs of it ever being fixed.