HUGE CONNECTION PROBLEM

[wasago]

after lastest patch i geting 90k error im on ps4 server ragnarog

[Runsell]

I am having Losses from ORUMCO Routers heading to SE Servers. No local problems from my ISP I had run lot of ping tests and traceroutes with no issues until I do it to SE Servers I see packet drops and latency when reaching ORUMCO.

[Askaniz]

I'm getting DC every 5 minutes since yesterday's night. 90k errors all the way.

[Jeza]

I am a member of one of the largest and most active FC's on Coeurl server, at least a quarter of our membership are having issues with lag and inability to log in to the game because of 2002 errors in the lobby and 90k disconnects in game. Lag is causing havoc with raids and events not to mention lower levels trying to run dungeons and do quests/leves ect.

That's alot of people from all over the world to suddenly, out of nowhere, have the exact same problem at the exact same time. There IS a problem and it is NOT user side. Our FC isn't the only one having this problem, we have been hearing about the exact same difficulty happening to FC's and players server wide.

FIX IT.

[Raist]

There IS a problem and it is NOT user side.

Exactly the point that has been made over and over in various threads on this issue. It is typically not a local problem, so when you call in and talk to the tier1/tier2 guys (that handle localized issues), they find nothing. The problem is typically upstream, between you and SE's ISP. As I tried to outline in another thread (complete with maps from the Tier1 ISP's mind you, this isn't speculation here), there are a limited number of pathways into Canada that we ALL have to use. It's like 8 from the US side, and one or two that come directly from from the UK into Canada. Everyone else around the world gets routed through the US, mostly through Southern California or the North East, then up through these limited paths into Canada. Note that all these routes are not used globally across all ISP's. There are a few Tier1 ISP's, each one controls a few of these routes. So from the US, you may have 3 optional ISP's as a routing partner, each one with only 2 or 3 routes into Montreal). If your route is bad, your ISP can either contact that partner for assistance or switch you to another partner.

For instance, there is a MAJOR trunk that runs from Tokyo to LA. If Cogent subscribers in North America want to get to/from Tokyo (as In SE Japan), they are likely to be sent through LA to get there. If there are issues with the lines in the LA area, it can dramatically impact ALL Cogent users in NA trying to communicate with Tokyo. Note that Cogent is a Tier1 ISP who YOUR ISP has peering agreements for connecting you to Tokyo--AT&T, TWC, Verizon, etc. typically do NOT run you everywhere on their network alone, but often use a third party to get you to someone else's network. Likewise, congestion can build up really quick around all the centralized exchange points like LA/San Jose, Boston, Chicago (hello, NSA scan we all go through), New York, Nova Scotia, Vancouver, Toronto, and Montreal--that's just a handful of the common choke points, there are others but the names escape me.

So, basically, when it goes bad for one of the big 3 peering/transit providers, it can impact entire REGIONS at once. This is why when you look at the maps for various services at the downdetector.com site, you will see problems pop up consistently in the same regions, even though you are dealing with different last mile ISP's like AT&T, Comcast, TWC, Verizon, etc. The "middleman" networks that provide the peering/transit to other people's networks are having issues along specific ports/routes and affect a widespread number of users all at once.

Think of it like the PS3/PS4 issues with PSN--when it has issues (or Sony takes it down for maintenance), the consoles have access issues because this game relies on PSN for access. Likewise, if the Cogent/PsiNet peer exchanges are having issues getting people through to Ormuco's VLAN, then EVERYONE assigned to that route has issues, but those that are using TATA may not because they aren't having congestion issues at that time.

This is why it is so important that you make sure you are dealing with at least Tier3 support when addressing the matter with your ISP. They are the ones that can provide escalation to conduct a proper investigation and contact the people that can adjust your route. It's not the guys at your nearby office that can swap modems/cables/connectors/traps/splitters.... this is a much higher tier of the network that needs to be addressed higher up the chain, typically in another city or state.

[Jeza]

How very unfortunate for FFXIV:ARR to be connected to so many unreliable ISPs that keep their players (from all over the world, east and west coast US and all over the map betwixt the two, Canada, England, Japan, Malaysia, Kuwait and all points in between) who play the game on consoles and computers, through Steam and not, from connecting to the game at the SAME TIME...

Gosh, what horrible, completely coincidental, luck for such a large company! They must just be heartbroken for all those poor customers that just happen to be connected to them through such a shoddy system.

Maybe they should contact other game companies who aren't having such a large scale and widespread issue and see if they can't find out why their games are fine but sadly it seems the interwebs doesn't want FFXIV customers to play their game anymore?

Poor SQ-ENIX, all those ISP's ruining their player's ability to play their game...maybe its a conspiracy! And here I am, all out of tinfoil to make a protective hat....

FIX IT!

[Raist]

How very unfortunate for FFXIV:ARR to be connected to so many unreliable ISPs that keep their players (from all over the world, east and west coast US and all over the map betwixt the two, Canada, England, Japan, Malaysia, Kuwait and all points in between) who play the game on consoles and computers, through Steam and not, from connecting to the game at the SAME TIME...

Gosh, what horrible, completely coincidental, luck for such a large company! They must just be heartbroken for all those poor customers that just happen to be connected to them through such a shoddy system.

Maybe they should contact other game companies who aren't having such a large scale and widespread issue and see if they can't find out why their games are fine but sadly it seems the interwebs doesn't want FFXIV customers to play their game anymore?

Poor SQ-ENIX, all those ISP's ruining their player's ability to play their game...maybe its a conspiracy! And here I am, all out of tinfoil to make a protective hat....

FIX IT!

Guess you missed that it is just 3 ISP's betwixt the US and Canada that are in control of the traffic. If level3 is having issues at even just 1 of their 2 trunks into Canada from the NY/MA area, then half of their customers are impacted. That means 50% of all players coming through the Eastern Seaboard via Level3 as their partner are impacted. That can be TWC, Comcast, AT&T, Verizon... any number of the last mile ISP's that are using them as their routing partner to Montreal.

In that scenario, said ISP's can either work with Level3 to address the peering issue, or switch to one of their other providers... TATA, who has 3 trunks from the US, or Cogent who also has 3. But... then again, if just one of their routes into Canada is having issues, then that is 33% of those customers. And so and so and so forth.

The ISP that you pay for internet service is NOT the one carrying you into Montreal. It is a THIRD PARTY. And the options for that third party are so limited that when one is having issues, a LARGE number of users are having issues ALL AT THE SAME TIME.

[neko5490]

Been having 90k since patch, every 15 min or so. Really annoying... On PS4 if it matters, Comcast, Florida here.

[Jeza]

I guess you missed the England, Japan, Malaysia, and Kuwait part....let's not forget Brazil and all the other countries affected in this.....or are they all connected through Canada too?

I feel like I am suddenly in a South Park episode debating with Cartman...BLAME CANADA!!!!!!!

The fact is, we have players from ALL OVER THE WORLD having this issue, all of them at the exact same time and nothing is being done about it...that is a problem.

SQ-ENIX is the root of this evil and no amount of blame passing will solve it, they...need...to....FIX IT!! NOW!

[Raist]

I guess you missed the England, Japan, Malaysia, and Kuwait part....let's not forget Brazil and all the other countries affected in this.....or are they all connected through Canada too?

I feel like I am suddenly in a South Park episode debating with Cartman...BLAME CANADA!!!!!!!

The fact is, we have players from ALL OVER THE WORLD having this issue, all of them at the exact same time and nothing is being done about it...that is a problem.

SQ-ENIX is the root of this evil and no amount of blame passing will solve it, they...need...to....FIX IT!! NOW!

If they are connecting to the Canadian Data Centers, then yes... they are potentially impacted as well. It can also apply to the Tokyo servers as well, depending on the route they are taking. Intercontinental connections can be even more susceptible to this issue because they may have even more limited paths to follow. Follow the path from Australia and surrounding islands for instance. Go look at the maps linked in that post if you aren't familiar with their routing woes. When it comes to interconnectivity between different companies networks, there is a big choking point at the peering/transit level. That is where the bulk of this instability has been found.

Oh, and as for the England reference... it was documented about a year ago to be an issue with a routing partner for the BT subscribers (think it was O2/Orange or something) where they had to work with their routing partner and SE to track down an issue with the handling of their traffic. Something to do with the packets getting a lower priority and getting delayed/discarded in route through the UK. It was posted at the forums for BOTH games, as well as on the ISP's forums as well. This is a very common issue that has existed ever since I started online gaming in 1996. There is plenty of precedent for it... you can find it related to a host of other games by different companies, even some hosted here in the US (LoL, GW2, LOTR, Blizzard's game library...).