I had a guy did same thing to me once
Basically, if you are not following GM's instruction to lock your house for public access
Anyone is free to go in; however, if the guy choose to stay there after you locked the house.
GM then has authority to remove him from the house interior and give him a proper warning.

If you choose not to follow GM's instruction, don't expect GM to do anything you think that is proper, because it is not according to their code book