To be honest, get 2-factor authentication. They can't log into your account unless they have your token. worst case is your account gets locked due to many attempts and you have to follow the instructions SE sends you.