Yeah it seems the beastman races are natives here, while the various races of man have came to Eorzea over time (A little ironic honestly!)

The fear from the elementals still comes up from Xenophobia though, even those who live within the Elementals' terms still face prejudice, see Keeper and Duskwight issues, I wouldn't doubt if that influenced a Gridanian's PoV in concerns to all incoming migrants.

The point about Ul'dah still stands though, how oft are immigrants coming in wealthy? Like when the PC get off the caravan/ship in the beginning, they're broke and don't seem to be in possession of anything really, so what's the difference there outside of plot?

I'm a bit curious on how someone would live in Eorzea without being at least some how having allegiance to any the city states. Towns outside of the main cities seem to be rather small and more like trade hubs over actual towns in most cases, and they all still seem rather firm in their association to their various city states.