Because they're "instanced duties" versus "one instance world" + "one instance duty.
Like I'm not sure if you've ever tried to visit someone's house when "everyone" was on and people inside had to be told to unsummon their minions so more people can fit in the interior instance.
The Island Sanctuary is basically the same as going to Eureka. It's not unique. The Island contains one tiny data blob of what the buildings are configured as, and uses an entirely separate copy of your character. Like go to the Island and yeet yourself off a cliff, you don't even take damage. That's because the "combat" isn't running. Where as both inside and outside the housing, you can take damage.
The problem is people don't really understand the difference between the various instances the game has. The game can have unlimited "interior" instances because they are only instanced when someone is inside it. Where as the exterior has to always be running since people have to walk to the house and see the exterior. Basically the same mechanism that they use for the Island and Combat instances is what they use for the inside of the building. The exterior meanwhile a special zone that is configurable.
Like the fairest way to deal with housing is to just collapse the housing system (Eg no wards) entirely, let players go up to any lot that they want, buy it, and then when "they" are in the housing area, they see THEIR house, and THEIR FRIENDS/FC's houses, in the places they expect them to be, and the rest of the lots just show the most recent public-accessible house the instance was invoked for. If there is a collision (Eg two friends have the same lot) then you click the placard and it swaps who's house you see in that lot. And if you don't know where someone's house is, search for it like you do now, right click on their name and go "go to house".
That's FAIR, but it devalues the housing itself. But now now lets people "move servers" without having to deal without losing the effort they put into their house. It goes with them. I just see why they don't do it, because it removes any "community" aspect a housing area might have. But isn't one of the reasons there are congestion issues in the first place, the housing itself? The interior itself is easier to deal with, the mechanism is already there. If we can have a cross-datacenter party finder and parties, we can clearly have a cross-datacenter housing interior.
An alternate scenario is to simply merge the housing system so the housing system itself is "another server" in the data center that is independent, so all housing exists, and you have to "transfer" to the housing server when you want to play on it. The housing server then runs none of the party finder, duty finder, and other combat instances. It's just a massive market that all the retainers run on. Any time you access the marketboard on any server in the DC, it is accessing this server. When you leave the housing area you are sent back to where you last were on the last server you were on. In this case, all the wards from all the servers exist as they are now, but they are just "one" housing area rather than dozens of wards per server world. This would unfortunately bring back the problem 1.0 had in the first place. Finding anything at all.
The situation we have now isn't perfect, but fit with the original server setup. Now that we can move servers and data centers at will, locking housing to just the home world creates a complicated problem for both the person wanting to keep their housing plot and the retainers on the market boards, and people who want to shop.
The original problem 1.0 had with the retainers is much the exact same problem we have with the player housing. It is impossible to find what you are looking for if you have to visit everyone's house on every world on every data center to find them.
Now I'm not proposing either solution as "good" solutions, only "a solution". What the fan base would prefer is to keep things as is, and play the lottery so they feel they earned that spot.
As for "why are houses auto demolished", I feel this is just the nature of the beast. By there being "real estate" in the exterior "instance", it costs money to persist (not "sub" level of moneys, but it does consume resources.) The interiors only persist when someone is inside. What would be nicer is for Square-Enix to actually "backup" the house if a player doesn't pay their sub, and when they come back, tell them which world has that space free, or plot that size of house fits on and offer them the option to rebuild it if they win the lottery or wait for a better location. In theory, every "ward" is it's own instance, but each plot is a data chunk itself, which is why they're so small relative to other maps in the game.
Housing and Retainers are excuses for players are "paying their sub" even if they haven't played the game. There are other things that seem predatory (like the app), which requires paying money to ... unlock half the chocobo saddlebag. Seriously? That's something you can cancel and not even remember you had it.