In regards to Garlemald, the point of distrust stems from the simple fact that the Garleans have never been left alone. They're unable to wield aether, which made them easy targets and they started out as peaceful farmers just looking to work to land. Yet bit by bit, they had their territory stolen and were forced to head to the frigid, inhospitable wasteland to the north. Despite that, they were still forced to fight against invaders and do whatever was needed to survive.

That hardened them - understandably so. Most people can't fathom the idea of nearly being wiped out and having to do whatever it takes to survive.

When they finally found themselves in a position of dominance, they began to reverse their fortunes - but they genuinely believed in the cause of bringing peace and prosperity to every corner of the world. It just so happened that their goal was actively sabotaged by a certain Ascian.

It's also not unreasonable for Garlean soldiers and civilians alike to suspect compromised motives from a former enemy showing up to 'help'. Let's not forget that Eorzeans exploited the Isle of Val as a way to get around Kugane's trade tariffs, actively sabotaging Hingashi's livelihood in the process.