Talked to my ISP, basically they said the problem is with the ISP's routing to the server's network.