Real life emergencies that draw you away from your PC should be rare enough for this not to be an issue.

If they're common enough to complain then you should probably play less.