"you would either need to sheathe your weapon, or cancel targeting"

This is already true right now for gamepad user. If I'm targeting a boss while he cast a turn-away kinda spell I will get hit by it even if I'm not looking in his direction.