This game has the absolute WORST targeting system I have EVER seen. And I've played some pretty bad MMOs before. For some reason it works based off what is in your camera's view. So you can be past an enemy, facing the one you want to target, but if you press Tab, it'll target the one BEHIND you, simple because it's closest, or something. Similarly, it'll cycle through random non-hostile enemies (or at least enemies not being attacked) waaaaaay off in the distance, as long as you can see them.

Honestly, the only way around it I've found kinda, is to face your camera completely towards the ground so you only see the enemies around you. It's not ideal, and definitely doesn't always work because the enemies may run out of that field of view, but it's the best I can come up with.