*We* see the Ascians for the obvious villains that we are because we are adventurers and we have the Echo. To your average person, I am under the impression that the Ascians aren't so obviously evil and so can smooth talk people into believing what they want.