The mystery of the targeting "cone" for tab targeting has been solved!
https://youtu.be/QzAVC-DNyEA
Simply, the "cone" is the camera cone you see in the mini map.
Any target within that cone will be selected via Tab or Shift+Tab (default settings)
The two targeting types for tab targeting were added in Patch 4.1 on Oct 9, 2017.
Patch 4.1 Notes (Full Release)
Originally, the settings were described as such:
Code:Target Type Type 1 (standard) Targets enemy closest to the player without considering depth in terms of the camera position. Type 2 (new) Targets enemies based on a cone shape originating from the position of the camera.