Casual gaming = playing "a little"
40M gils for the "worst" house = farm hard.

Yes, those prices "kill" the casual gaming. But, for myself, i prefer HARD content, than easy things you have to do hundred (and more) times to have anything.
I would prefer a really hard fight you have to do with your FC to buy house than collecting 40M and more gils for a garden shed