There are multiply options - in the player settings you can modify the selection filter.

While in group it's the best to use F1 - F4 to select 1. - 4. party member and F11 the closest mob you're facing.

Alternatively just plug in the USB mouse and use it for targeting (or anything else indeed) instead. You can also use the "virtual" mouse for that (L1+R3 to toggle virtual mouse on and off).