Yeah it is a little annoying.

If I am using my headset for chat and forget to unplug it the next time I play the audio comes out the headset causing me to have to restart the game so its out the speakers.