I have never had this problem. Maybe issue with route from ISP to Server, you can try having your PC directly connected to your modem to rule out network issues on your end.
My Location: Ottawa
My ISP: Rogers