If a VPN improves things, than you have a routing issue your ISP needs to address. A VPN does nothing with the server...it bypasses your ISP's routing policies and also may send you across different nodes entirely in an attempt to get you on a more optimized route. You have a golden opportunity to demonstrate the problem to your ISP... need to capitalize on that. You may improve things for a lot of people if it leads to them fixing a flaky node or updating their peering agreements to clear up problems for their users.