I don't think Garleans are an evil race but their current rulers most certainly are antagonists. Later we may find a more moderate high ranking Garlean we can work with but right now its bad guys who hold the reins.
I would also point out that in general the people suffering from the actions of the Ishgardians are mostly other Ishgardians. Very few people knew the truth. Further Ishgard's situation is quite different in that it was in a war it couldn't get out of. Nothing short of Nidhogg's death and the Horde's defeat would end the war cause Nidhogg had no interest in every letting Ishgard off the hook even though the one who committed the sin had been dead for a thousand years.
Everything we have seen of Garlemald's expansion has been born out of a hunger for power, hubris and belief in Garlean superiority.
Gaius did have his good points but he also allowed atrocities to happen under his governance by his own Legion. You really think he had no idea where his conscripts were coming from and what was being done to the populous? If he didn't it completely undermines any credibility he had as a leader.
Also where has Eorzea used conscription? Gridania's forces are the united God's Quiver and Wood Wailers, Limsa Liminsa's armed forces mainly are formed from the former fighters of the various fleets and Ul'dah's Grand company is entirely made up of mercenaries and gladiators originally. The only case you could argue for a country using conscription is Ishgard and even then it doesn't seem extremely common.
Even if they did use conscription in none of the city states forced prostitution government sanctioned and in none of the city states are the governments guilty of taking youths, brainwashing them and having them kill their family members.
Do remember that people in occupied lands don't even have the rights of Garlean citizens.
The last major war between the city states of Eorzea was the Autumn war 100 years before the current timeline. No one was trying to invade anyone for a really long time in Eorzea.
Eorzea is very morally grey. There is no question of that. The story of what happened leading up to the Garlean invasion of Ala Mhigo showed dark stuff happening.
The difference is Garlemald is the aggressor. There is nothing that changes that. They are the ones attacking and conquering. I think it would be a huge stretch to argue that all the conquered territories the Garleans have taken somehow antagonised Garlemald into having them attack.
If a guy walks up to you in the street who you barely know and have done nothing to and starts swinging their fists at your its pretty hard to argue they aren't the ones in the wrong. It's even harder to argue you are in the wrong for fighting back.