You probably are worrying a shade more than most people would but if shutting your pc down when it starts to get warm makes you feel safer then you aren't hurting anything (unless you are excusing yourself from parties to restart your pc...)

In short if it helps with your own peace of mind to be extra careful, go for it.