You are sort of answering your own question (notice you are also not using Level3 there? Don't see xo.net either...).
WoW is by no means immune to the exact same phenomenon you are describing for this game, so please stop pretending like it is some perfect contrast to this game. The fact that you appear to have solid sessions to their servers and also have solid game play with them is more or less proving the point--you have good routing to them, and that is reflected in the response times. The internet is full of complaints over bad latency knackering up WoW just like it does for XIV, XI, GW2, LoL,....even Netflix and YouTube for heavens sake. There is a reason Blizzard has that post detailing how to test for latency after all...many other services will have similar articles/tools posted as well for conducting tests. It is because it is a KNOWN problem that must be investigated and ruled out of the equation.
There are known address groupings that habitually demonstrate stability issues associated to the very symptoms you are describing. Level3 is not the only ISP that has been flagged for it in the past, but it shows up quite often in forums across multiple services---including WoW. And specifically those car2 lines in the Montreal area are sort of notorious for it.
Go ahead and google latency issues for WoW if you don't believe me. I'll even provide a link for a very simple search for you:
https://www.google.com/?gws_rd=ssl#q=latency+issues+WoW
No matter what is going on at either the client or server side--if you have a flaky route, a client-server based service is going to be negatively impacted. PERIOD. If there are signs of issues in routing, they need to be addressed, otherwise you are going to be open to the negative affects that can be triggered by those problems. This has been demonstrated often in these forums as well as elsewhere in the past. Try hosting a space combat skirmish/dogfight with north of 130ms latency for everyone else, than do it again when the latency is 60-80 for everyone. The players will see and "feel" a marked difference in accuracy and such. The only thing that changed is the latency.
Oh, and by the way... Time Warner has switched me back to peering with TATA again for getting to Ormuco. Just saying....routing policies can be amended to address a lot of routing issues once they are properly investigated.