You are correct on "I'm sure there's a technical thing with that". Namely, the inn room (or Grand Company squadron barracks) seems so far as I can tell to only exist on your client, not on the server; this is why no one can come into your inn room with you, or visit your barracks. It's also why there can be an infinite number of inn rooms or barracks in use without any issue; they don't, strictly speaking, exist from the standpoint of the server.
Housing interiors are dynamically instanced, and the 'door' acts as a queuing system. If you're the only person in the house, when you teleport out (or enter a duty queue, or log out), the house—as an instanced zone—ceases to exist, thus there's nowhere to return you to when you go back.
Worse, if you teleported into your house interior on teleporting, but you actually wanted to get to the yard, it would require spinning up an interior instance, loading all the customization data, loading you into the zone... only to have you click on the door and exit immediately to get to your yard. Conversely, the exterior is shared by everyone; that instance is almost certainly already there to drop you into, and if you did need the interior of the house, you can walk up to the door to go in.
They did kind of do a workaround for this for Island Sanctuary; if you watch carefully, upon leaving a duty and being dropped back into your Island Sanctuary, it actually loads you into Moraby Bay first, then basically seems to functionally "use the NPC" to get you back to your island (and restore your location in doing so). But that works because so far as I know you can only be returned to your own island sanctuary, not a friend's; it doesn't need to think about "you were in this spot in the house at this plot on this ward" like it would with housing and then (functionally) teleport you to the door and use the door and then move you where you were. Instead, it can just be a "I was on my Island Sanctuary, [yes/no]" flag.