I sympathize with your situation.
Housing, as they've implemented it in this game, is terrifically resource-intensive. SE knew that all along, but they also knew that players really wanted housing and so they rolled out a limited feature they could iterate and expand, all while trying to get a better handle on the associated technical issues.
The first housing was limited to free companies. Players had been asking for individual housing since forever, but they told us they simply didn't have the resources to pull it off. There were originally five wards and no subdivisions, and these sold out over a period of some months. Players responded positively, but everyone wanted to know when individual housing was coming. They added a sixth ward of free company plots a couple patch cycles later, as well as personal rooms in the free company houses. Everyone continued to ask after individual housing. It wasn't until the patch after that that they opened up housing to individuals, adding two more wards at first, and then subdivisions shortly after that (doubling the amount of available land). I wasn't playing at the time but I am sure there was a feeding frenzy the moment the servers went live, leaving us more or less where we are today.
The simple fact is that they cannot implement housing (in its current form) for everyone who wants it. Square Enix had an overly ambitious vision for what housing should be, and they are committed enough now that they don't really have any choice but to keep applying band-aids to the problem: providing players with alternative points of access into the housing system (personal rooms, "mansion" housing), reclaiming unused houses, and adding new housing as they can. Ishgardian housing will increase the number of available houses by fully a third, or even more if they couple it with any sort of expansion to the existing districts. Everyone knows that isn't going to solve the housing crisis, but it's something.
Perhaps with the benefit of hindsight they would have done it differently. I cannot say. I do know that the prohibition against buying and selling housing is meant to protect us, not to lock new players out of the system; they are unsupported transactions based on unenforceable agreements, and they put SE in the position of arbitrating "he said, she said" disputes including agreements that might have been made outside the game (and they probably open the door to RMT transactions as well). Take that for what you will.