Being able to go wander inside personal houses is silly, but being able to actually enter public buildings, stalls and whatnot like you could in real life, what's the problem with this?
Although I'm saying this as someone that literally loves spending half-afk time in those nifty S9 apartments...