Racial stats are so small in variance as to be completely inconsequential. If it REALLY bothers you for the absolute min/max potential then switch, but if the race itself is more important to you then don't worry about it. It really, seriously isn't going to affect your gameplay that much. You talk about being "at a disadvantage", but if a Dunesfolk Lalafell tank can do just as well as a Sea Wolf Roegadyn tank in practice and ease of content (and they can), then it clearly doesn't matter all that much.