My point would be that a large number of other players are not encountering this "problem". Maybe see if there are other Australians on your server that can verify? It could be that specific server, etc.
You likely get routed through Texas then to Montreal (~3000KM) which should add on 60-70ms, plus a little bit depending on how many exchanges it goes through. You can do a tracert to find out.