I'm on PS4 so I never had any of that. I have noticed that when I enter a housing zone that the PS4 fans make more noise than usual meaning it's working harder..
I think what they could do is only load things that's on your screen. It may not be pretty when you walk up to a house and see it pop up in front of you but it's a good way to save energy.
