Details about Server issues

[Brevitic]

Today there are issues with connectivity to the Montreal datacenter.

The issues are located somewhere in the routing and peering between Level3 and Ormuco Communications

The node between ORMUCO-COMM.car2.Montreal2.Level3.net[4.59.178.74] and [19.34.76.2] has been failing to respond all day.

Significant congestion can be seen upon entering the level3 network from as far west as Seattle at ae-32-52.ebr2.Seattle1.Level3.net [4.69.147.182] where connection latency goes up 300%

[seannykun]

IT'S BROKE

[Mega64]

Try unplugging and replugging the modem.

[Sonoran]

Try unplugging and replugging the modem.

Umm a lot of people are having this issue, its not all of our routers

[seannykun]

Try it anyways.

[Namnesor]

No I don't have any big downloads running MOM

[Burrmanchu]

Servers are fucked.. has nothing to do with this guys router. Jfc people. D/c'd many times in last hour. Expert roulette run ruined and lost a T2 drop 20min prior... FIX YOUR SHIT SE I HAVE COIL TONIGHT.

&@Mega64: Did you read the OP? The guy has it pinged and nailed down to a specific communication error and has a half page of techinical mumbo jumbo pinpointing latency vs areas on the globe. Did you really need to tell him to reset his freaking modem? People never cease to amaze me. You must be that guy I always have to talk to at Comcast.

"IS IT PLUGGED IN?"

[Enepttastic]

Servers are fucked.. has nothing to do with this guys router. Jfc people. D/c'd many times in last hour. Expert roulette run ruined and lost a T2 drop 20min prior... FIX YOUR SHIT SE I HAVE COIL TONIGHT.

&@Mega64: Did you read the OP? The guy has it pinged and nailed down to a specific communication error and has a half page of techinical mumbo jumbo pinpointing latency vs areas on the globe. Did you really need to tell him to reset his freaking modem? People never cease to amaze me. You must be that guy I always have to talk to at Comcast.

"IS IT PLUGGED IN?"

You do realize there's actually at least two good reasons tech support asks you those "stupid" questions?

1. Any troubleshooter worth a damn will make sure and check any potential cause of the problem, often starting with the simplest to confirm first. Since the person at the other end of the line may not be able to check if your modem/router's plugged in, they gotta ask.

2. The sheer number of calls I had in the short time-span I did ISP tech support that dealt with a bitchy customer who rants and raves about their service being interrupted finding out that the modem was, in fact, NOT plugged in was staggering.

Another reason is that they HAVE to ask those stupid questions else the rep is going to get in trouble for potentially wasting time, not just the companies but yours as well.

[Burrmanchu]

You do realize there's actually at least two good reasons tech support asks you those "stupid" questions?

Do you realize that anyone with a basic level of human intelligence would see that the OP was -far- beyond unplugging a modem? This was not a "ITZ NOT WERKING" post. Just read it. Also your job at a ISP has nothing to do with a random forum person. They aren't required to do anything. Thanks tho.

[Raist]

Do you realize that anyone with a basic level of human intelligence would see that the OP was -far- beyond unplugging a modem? This was not a "ITZ NOT WERKING" post. Just read it. Also your job at a ISP has nothing to do with a random forum person. They aren't required to do anything. Thanks tho.

/sigh

Fair warning... moving into Dennis Miller mode.... trying to resist it... but... can't ... hold-on....

[rant]

From page 26 of Cogent's User Guide, available here:
http://cogentco.com/files/docs/custo...user_guide.pdf

In the event that you have lost service, please have the following information available when calling Technical Support:
-
For incidents in which you have lost connectivity to the Internet or external sites:
Can you ping the gateway on your network to which the Cogent line is provisioned?
If there is no response from your gateway, please ensure that your equipment is receiving power and that all cables are properly attached. If possible, power-cycle your equipment.
If there is a response from your gateway, please specify whether or not you are able to ping sites on the Cogent Network such as your default gateway or 66.28.0.45
(res1.dns.cogentco.com).

(colorized the relevant line myself, not done in the original document)


Just saying the process is troubleshooting 101 in the IT industry, and (as noted in previous posts) the simple process of rebooting devices sometimes resolves an issue and sometimes is required to implement configuration changes that may have been made in order to address an identified problem.

In the case of an ISP, sometimes such changes are made at the servers that give out leases and configuration files without being actively pushed, but instead get handed out during the next scheduled renewals. In the case of my ISP (TWC), they have frequently made changes to config files to upgrade services on a broad scale (like bumping basic internet from 10Mb to 15Mb) on their TFTP server but do not push a reboot to everyone's modem to affect the changes immediately. They instead will insert a notice on the next bill advising people of the change and telling them to power cycle their modems to receive the updates. This allows them to exert some level of control over how the rollouts take place so they can somewhat monitor the impact on a regional basis.

So no... someone's experiences working with an ISP (or networking in general in the IT industry) actually DOES have something to do with this thread, as they have very credible knowledge and experience with the particular issues at hand. If not for my limited experiences working with TWC during their local DOCSIS 1.1 Beta rollout of Cable internet back in the 90's and then later for a small MS/IBM partnered software firm years ago when TWC was rolling out their local 2.0 Residential and Business Class internet (helped sell, plan, and implement new drops to local businesses and assisted in troubleshooting networks then just as I did during their Beta phases), I would not have the basic understanding of how all this works in order to find documentation to present to the forum to hopefully better educate the masses on how to properly address the well documented issue and to actually get it addressed by the responsible parties.

This has all happened before, and it will all happen again. It has happened time and time again for multiple games/services going back into the 90's, as well as for various content services including but not limited to Netflix, Hulu, Amazon Prime, Twitch, and JustinTV just to name a few from recent memory. It's even recently been documented as an issue for reaching content for Google/YouTube. It's amazing how much information you can find via Google if you put your mind to it.

And in nearly every case (barring instances where internal issues were found related to things with bad packet headers or actual server capacity/stability problems), it has been identified and corrected by the intermediary ISP's that were being used as routing partners--which has been identified by the OP (Level3 is in no way affiliated with XIV or SE for that matter). This particular issue that has been identified in the OP is clearly something that needs to be addressed by the ISP's at the Tier3 or higher level, and in some cases it ultimately may indeed require the power-cycling of a modem (or at least a router or client attached to it) to resolve it.

In my own past experiences with not only FFXIV, but also FFXI, Diablo (1 & 2), Total Annihilation (and other games using Microsoft's Game Zone at the time), Descent: Freespace (1 & 2), and various games on both my PS2 and PS3, and more recently my Roku's (who were having issues with throttled content because of a routing partner's shaping policy), the fix has often required me to at least reboot my network hardware or at least conduct a reset of my connection config (either by a ipconfig /renew or a full out netsh int TCP reset). In each case where such a reboot was required it was because TWC/RR was basically changing my modem's "home" of sorts as a quick and dirty fix to get me rerouted. It basically was "spoofing" my geolocation so that I would get sent along a considerably different path to avoid stability issues until a permanent fix could be put in place along my normal route from the Carolinas. After rebooting and getting assigned a new route, I would be asked to test and verify if routing was better, and if necessary we would do it again until we found a viable path.

Back during the holidays I was going through this process about every 3 weeks with TWC trying to stay ahead of the congestion issues... and guess what? It kept me playing while everyone else was continuously flaking out and unable to play, coming here to just scream and complain... blaming it all SOLELY on SE. But, you know the really interesting thing? Those who listened to the advice of those trying to explain this process and how to remedy it that actually took the advice and properly pursued it--they got results. In one such case, a small group in the UK (O2/BT Orange customers) actually fingered an issue that was impacting an entire region over there and managed to get the ISP's in touch with SE about an issue with their packet headers. It was all over their forums back then, just like TWC currently has a firestorm on their forums to address their identified internal routing problems (we've just about got things resolved for the most part with their partners). This ultimately resulted in a change that addressed a large part of the throttling issue that was impacting people worldwide last year, and because we've recently identified and affected changes addressing a big problem with our headends here in SC, we've fixed issues for a large portion of the eastern half of our state--hopefully more fixes will come with new channel alignment rollouts that are on the way (haven't been able to drag any details out of them about speed increases, but they are considering channel realignments because of identified ingress issues between 573 and 615MHz.).

All because some people familiar with the ISP's and how the system works got involved and got a group of people to lean on their ISP's to affect a change locally--and I assure you , to properly implement some of those changes, a LOT of modems had to be rebooted.

So, I respectfully advise that you be careful about belittling the advice of someone connected to the industry who actually may have pertinent knowledge of the situation and is actually providing relevant advice.

[/endrant]