Imagine a vertical line in the middle of your screen dividing your view in half. The first target to the right of that line (at any range) will be the first selected target. Press again and the next to the right will be selected, and so forth till you run out of targets on the right side of your screen. At this point the first mob all the way to the left on your screen will be targeted.

How to manipulate this:
1. Pan up with your camera. Less distance on your view = less targetable things.
2. Zoom in a bit. Less area on your view = less targetabe things.

I do agree, however. There should be an option for near-to-far tab targeting.