The Empire is probably on the 'bad' side of the spectrum but Eorzea is not a world where it pays to be a good guy. Realistically, the only country that has a prayer of survival is Giridina and that's because their neighboring opposition is weak/peaceful (Sylphs). Although the Beastman tribes are more or less on an existential conflict with the various humanoid races of Eorzea, and that defense against tempering is extremely rare, the Beastman tribes somehow have not completely wiped out a bunch of feeble opposition that can only get weaker over time due to tempering. Realistically the Empire ought to be more woried about what if Eozrea got wiped out by a Beastman tribe and everyone turned into thralls of any given Primals as opposed to any threat posed by any humanoid faction of Eorzea. The Empire might be evil, but being evil is better than being dead, which is what should've happened to LL and Uldah given they shown no ability to defend against a threat that is the classic 'each guy you lose they get stronger' type. From what I can tell the Beastman tribes are at least equal in power to the major factions (Gaius didn't try to wipe them out until he had the Ultima Weapon so they're clearly a serious match even for the Garlean Empire) and throw in tempering, there's no way the Beastman tribes can lose.