You probably don't have a static IP with your Internet provider and so it has the chance of changing, even more so if you regularly switch off your router.

If you add the security token (app or physical) like Silverbear mentioned above it'll solve all this as that overrides the IP check (as only the owner should have access to generate the one time password).