I think the access limit to house has been proposed a long time ago. I don't know why SE hasn't implemented it yet. Maybe the database requirement might be too big?
As of now we only have Private or Public, which is not good. It's like Black or White, there's no such thing as Gray.