Originally Posted by
Cleretic
Also, at their peak the Garlean Empire had all of Ilsabard, most of Othard and Ala Mhigo under their control. That's a lot of countries, and given the Empire's spread only lasted like 70-ish years tops (the first invasions were in 1515, Seventh Calamity in 1572, nebulously 5+ years after that to Endwalker), the Garlean people didn't have a huge chance to proliferate into the lands they were now able to occupy.
At the scale the Empire was at, in the time they had to get there, it would've been really hard (and probably fairly morally terrible) for them to not be the minority population in their own greater empire.