I got this issue today and fixed it with VPN. (Used Spotflux which is free.)
The fact that VPN makes the problem go away is practically proof that this is an ISP throttling issue and not a server issue.
That and the fact that the server works perfectly fine for everyone else of course...