Horrible lag on Balmung the past two evenings.

[Raist]

Your ISP can resolve it because the breakdown is at the exchange point between Level3 and Ormuco. That last hop before the private IP is Level3, the first hop after that is Ormuco. Sometimes the fix isn't a very immediate one. They may need to request bandwidth increases and such...install additional port cards maybe, maybe even remove an existing router and bring in a more robust one. These things can drag out for a while, depending on how bad the pissing contest gets. On the other hand, your ISP can alter your routing much quicker to try to avoid the troubled hop(s).

Your ISP has the resources and ability to rather quickly change you to using a different peer (at least any of the larger ones in the US will), or possibly change the pathing so you come in on a different vector (you could go NYC to Montreal, or over towards Buffalo and slingshot Toronto--might put you on a different port coming into Montreal even with the same ISP). Here is a breakdown of the peering options available for connecting to Ormuco:

AS3257 	Tinet SpA
AS3356 	Level 3 Communications, Inc.
AS701 	Verizon Business/UUnet
AS174 	Cogent Communications
AS6453 	TATA COMMUNICATIONS (AMERICA) INC

TiNet is more foreign country focused, but the other 4 making up roughly 71% of their peering are major providers throughout the US and beyond.

Your ISP is free to peer with anyone of them and most likely peer with at least two of them already, if not more. My ISP (TWC-SC) peers directly with TATA and Level3 right here in the Carolinas, and one hop further to another trunk just one state over (Georgia) picks up Cogent. They frequently flip me amongst these three ISP's routinely.

Literally...in virtually no time flat I can get flipped from going to Raleigh-Durham or Charlotte or Atlanta to any one of the other two. Seriously, at the drop of a hat, they can cycle my modem remotely and I am updated. I have had the modem cycle in the middle of Expert Roulette, logged back in before I get booted, continued the run and then checked my connection and find I have switched peers and am now running a different route to Montreal. Not kidding, I have actually had it change twice in one night...TATA when I got home, ate dinner and came back and was on Cogent, and before I went to bed they flipped me back to TATA again.

It simply should be one of the quickest ways to get an alternate route and <hopefully> avoid a troublesome segment. Another is to use a flexible VPN service with tunnels all over the place to choose from. Both approaches can accomplish the same means to an end. You are flopping around on Level3...change the route so you come in on either someone else's lines or at least use a different path on the same ISP (like those car2.Montreal lines of Level3--routes with those routers regular show up in posts where people are having issues).

Your ISP may be your best advocate for resolving routing issues if you find them.

[Gyson]

Your ISP can resolve it because the breakdown is at the exchange point between Level3 and Ormuco.

I think we're defining "resolve" differently. You're suggesting ways for me to avoid the problem, but that doesn't mean the problem is fixed. It's still there, just no longer impacting me (until eventually it is). Where as I'm asking Square to get involved and put pressure on getting the actual problem fixed (whether it's failing hardware, bandwith issues, much needed upgrades, etc).

I appreciate the suggestions, but if you ever dealt with AT&T in my area you would know that what you're suggesting is a lot easier said than done. They have and will make me go through hours of pointless troubleshooting for a problem that has nothing to do with my home before the lift a finger on their end to do anything.

[Raist]

Customers need to be letting their ISP's know when their routing policies aren't working well enough so they can investigate and make the needed changes. Otherwise, you will just continue to get the same-old crap they've been dishing out for well over 20 years now. Not saying it won't take some persistence, but the Tier3 support team can and should be able to resolve it somehow. Maybe there needs to be more people pushing on the matter? Just saying...if you don't let them know some useful details about the problem, don't be surprised if it isn't getting fixed.

We are regularly going after TWC: forums, email, some use twitter/FB---and things gradually improve over time. When I say "we", I mean the TWC customers in general, not just players of this game. The problem persists in various regions for various services, but statistically it seems to follow a pattern. Level3 and Verizon's branding tend to be the worst offenders(they've gobbled up a lot of networks, and some still report the old names), TATA tends to be more reliable, with xo and cogent somewhere in the middle (though, xo tends to get ugly sometimes out west).

I come across the same Ormuco subnet for my exchange into them, but I am coming through TATA and hit 192.34.76.2--and don't experience such issues there. So either the congestion is creeping in somewhere within Level3's network, or there is an issue with the exchange itself. Level3 has a long history of issues in that area, especially with the Montreal car2 segments, so it's pretty easy to turn the eye towards them.

It may simply be because too many people are using Level3 when they could be using someone else, and it leads to a bad bottleneck. So, in effect....getting AT&T to lean on Level3 to investigate or otherwise route differently for that area could in fact fix the problem--or at least work towards fixing it. Either they encourage Level3 to beef up the weaker segment(s) somehow (be that internal upgrades, or revisiting the peering arrangements at either end--AT&T or Ormuco), or they route around them and start using someone else...which in itself might encourage Level3 to become more proactive in the end. Either way, the customer wins as the data begins to flow more reliably...which actually is resolving the problem.

But unless enough people are voicing their concerns to inspire someone to get off their duff and look into it more closely, they are less likely to take any proactive steps to remedy the problem(s).

Oh, and yes, I have worked with AT&T in the past. Granted it was about 10 years ago with T1's and ADSL (used to work on implementations for a local hardware/software firm)...but the process hasn't changed much. Now just as then, you seem to get better responses if you use online contact versus the phone. But there is one additional tool now that seems to help a lot if you can use it--we now have social media to put more pressure on them.

The technicians basically use the same tools for rudimentary testing (some of which we have at our disposal as well) as well as more advanced testing, talk to the same departments. They'll want someone to check the DMARC and blah blah blah. Let them do their song and dance, swapping the modem, new cables or new ends on them, checking signal levels--all to prove what you have already confirmed, that locally things are good and the problems are upstream from the location.

Then you can demonstrate the problem to them on site--use of a VPN for a quick comparison of a good and bad route is very handy for this by the way. After you finally get someone to actually witness the problem in action, then they can escalate things past Tier2 and get Tier3 involved, which is who needs to be looking at it, and then escalate things from there.

It isn't just AT&T that pulls that mess. They ALL do it.

Here are some sites that might help you get some help resolving things more directly with AT&T:
https://downdetector.com/status/att (has links to AT&T support portal, Twitter, Facebook)
http://www.dslreports.com/forums/all (another forum where you can get help..some of them are monitored by the ISP's as well)

[Adire]

I think we're defining "resolve" differently. You're suggesting ways for me to avoid the problem, but that doesn't mean the problem is fixed. It's still there, just no longer impacting me (until eventually it is). Where as I'm asking Square to get involved and put pressure on getting the actual problem fixed (whether it's failing hardware, bandwith issues, much needed upgrades, etc).

I appreciate the suggestions, but if you ever dealt with AT&T in my area you would know that what you're suggesting is a lot easier said than done. They have and will make me go through hours of pointless troubleshooting for a problem that has nothing to do with my home before the lift a finger on their end to do anything.

I can second this. I'm also on At&t (and in the south unfortunately). For some reason the past few days, almost exclusively when playing FFXIV, my ping and connection health go insane. Something about FFXIV is destroying my connection lately.

After lightning struck a telephone pole a couple of years ago and dealing with shaky internet during raining ever since, the only "help" I've received from At&t has been 4 new routers.

That's At&t's solution to everything. Send out new routers. If I call them about this issue, they will put me on the phone with someone who barely speaks English and tell me to reboot the router, when it inevitably doesn't work (I've tried it 6 times in the past hour actually), they will send me a new router, and it still won't work.

At&t just doesn't care. I don't know about where Gyson lives, but here, where they have basically no competition, they don't have to care. That's why they're still charging an arm and a leg for internet that is scarcely better than what third world countries are getting while the rest of the modern world is getting cheaper, faster internet. I'm paying more for my 6mb/s internet than some of my friends are for internet that is 10x faster.

We have called at&t, we have contacted them in every way possible, been polite, raged at them, threatened to quit doing business with them, but they know better.

[Raist]

I can second this. I'm also on At&t (and in the south unfortunately). For some reason the past few days, almost exclusively when playing FFXIV, my ping and connection health go insane. Something about FFXIV is destroying my connection lately.

After lightning struck a telephone pole a couple of years ago and dealing with shaky internet during raining ever since, the only "help" I've received from At&t has been 4 new routers.

That's At&t's solution to everything. Send out new routers. If I call them about this issue, they will put me on the phone with someone who barely speaks English and tell me to reboot the router, when it inevitably doesn't work (I've tried it 6 times in the past hour actually), they will send me a new router, and it still won't work.

At&t just doesn't care. I don't know about where Gyson lives, but here, where they have basically no competition, they don't have to care. That's why they're still charging an arm and a leg for internet that is scarcely better than what third world countries are getting while the rest of the modern world is getting cheaper, faster internet. I'm paying more for my 6mb/s internet than some of my friends are for internet that is 10x faster.

We have called at&t, we have contacted them in every way possible, been polite, raged at them, threatened to quit doing business with them, but they know better.

You need escalation to Tier3. Still stuck at the lower levels of support. Again...usually fair better with online support options for this. The call centers of most companies are more along the line of generic helpdesks following scripts.

[Adire]

We've tried everything, including online contact, believe me.

Corporations with pseudo-monopolies in an area just aren't very customer friendly. Why spend the time and resources and money when you don't have to and customers will still stay because they have limited options?

[Gyson]

I just wanted to comment and say that this issue is still ongoing (day 4). I know threads here have habit of being marked resolved and closed if updates aren't being given.

====================== PING ======================

Pinging 199.91.189.139 with 32 bytes of data:
Reply from 199.91.189.139: bytes=32 time=65ms TTL=42
Request timed out.
Reply from 199.91.189.139: bytes=32 time=66ms TTL=42
Reply from 199.91.189.139: bytes=32 time=66ms TTL=42

Ping statistics for 199.91.189.139:
Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),

====================== TRACE ======================

13 57 ms 57 ms 56 ms ae-11-11.car2.Montreal2.Level3.net [4.69.141.1]
14 56 ms 56 ms 56 ms ae-11-11.car2.Montreal2.Level3.net [4.69.141.1]
15 50 ms 54 ms 50 ms ORMUCO-COMM.car2.Montreal2.Level3.net [4.59.178.74]
16 67 ms 66 ms 67 ms 192.34.76.10
17 * 67 ms 66 ms 199.91.189.242
18 66 ms 65 ms * 199.91.189.139
19 66 ms 66 ms 66 ms 199.91.189.139

Hops prior to #13 are in the 10-25 ms range with no issues.

[RocheKat]

I just wanted to comment and say that this issue is still ongoing (day 4). I know threads here have habit of being marked resolved and closed if updates aren't being given.

Worst part is most people are not on these forums complaining. My servers facebook page got 10+ replies after someone asked if anyone else was lagging... and there isn't even that many people on that page.

[fallenalexiel]

I am also experiencing this issue. It is exclusive with FFXIV, and my provider is att as well.

Edit: Ultros server

[Ratham]

I have terrible lag on Exodus since the update this week. The game is pretty much unplayable outside of running around or crafting. It can't be an ISP issue if it's suddenly hitting so many players with different setups and ISPs, this is an issue Square created. All other games still work lag-free for me.