I don't see any of the primals as being evil, even with the brainwashing. They simply provide their tribes with the power that they yearn for. Power, without cost, is corrupting.

This corruption makes them more like their Primal, and less like themselves.

The Sylphs, as an example, are described as mischief makers. One and all. Ramuh's influence made their Violet kin isolationist, defensive, and without remorse. They do mischief without mercy, so they often take it too far, all in the name of defending the Sylphlands. They try, even to the point of kidnapping and enslavement, to bring unturned sylphs into the fold, because they're drawn to community as trees and grass to meadows and forests.

This does not make them 'evil'.