Much of the allegations against the Garlean army come from people who have escaped from their held territory, which is to say, from Ala Mhigo. That city is held by the XIVth Legion, which, far from being some rogue soldiers, is (was?) one of Garlemald's foremost and most-celebrated military forces*. Everything that's leveled against them - implied prostitution (not rape, actual prostitution), forced conscription followed by brainwashing via brutality, etc - speaks to ongoing military policy against a captured population. Furthermore, they've held the city for a long time - it was the first foothold in Eorzea, pre-dating van Darnus' arrival from Othard by a significant amount. And speaking of Othard, the picture painted of the Doman occupation by its refugees isn't exactly brighter, which would imply similarly brutal tactics being employed by the other conquering Legions. Now, these are obviously biased accounts, but nothing has actually been presented yet in-game that gives another view. That could obviously change, but until it does, the impression that the real-world storytellers want us to have of Garlemald-as-conquerors is pretty clearly not a positive one. As has been so astutely pointed out before in this thread, this game hasn't shied from giving shades of grey to its world and story before; if the dev team wanted us to view Garlemald with shades of grey, these accounts would have something balanced against them. We'll have to see as time goes on.
(As for citizenship, much of what we know comes from the Garlean hierarchy books in the Great Gubal Library, easily the most un-biased accounts available. We know that they don't afford the same citizenship considerations to the people of conquered territories, as Rome did; instead, they're fodder for the Garlean war machine.)
---------------------------------------------------------------------------------------------------------------------------------------------------------
*Yes, I know the XIVth Legion is effectively rogue post-Calamity. I don't think that's especially relevant; Gaius never lost his loyalty to the Empire, even after they abandoned him, and anyways they were occupying Ala Mhigo well before that.