The more I think about unspecified targets the more I think it might be possible to just use an enmity list if there is no auto-attack target... in other words the spell would target whatever has the highest enmity... if there is no target with enmity on the player or the player isn't targeting a monster then the spell can say something along the lines of "Reflect failed to take effect" and fizzle. This might or might not be a viable solution but it would be an easy one for the players to understand.

Also if you can cast Reflect on a monster as well I think this will be the second greatest spell in FFXI history (right after 99999 damage Cures).