Regarding Reports on Recent Connection Problems with the NA Data Center

[Rommie]

I've been hit by this the last night and tonight on Zalera. During EST roughly 8pm to 12am I'll get spiked and occasionally 90006'd. It's been fine at other times.

[Epiales]

Same issue here, along with two of our friends but not everyone in our static. Getting really obnoxious. We're on Coeurl. All three of us were getting 90002 errors at the same time yesterday, and today we're getting lag spikes randomly but at different times. All three of us have Comcast, but one isn't in the same state for what it's worth.

[Bluefve]

Happening to me all day now.

- When the disconnects started occurring: Noticed as of 5 PM PT, still happening.
- Platform (PC, PS4, Mac): PC
- Any error codes or messages you receive upon disconnect (if any): No disconnects, but horrendous lag. Stopping and zipping around.
- Type of connection (wired or wireless): Wireless
- Other programs running while the game is playing (if any): No
- Frequency at which disconnect occurs: Constantly
- Where you are booted to after disconnect (Desktop, Start Screen, Character Select): No disconnection, but noticeable in Quarrymill and in the Palace of the Dead in particular

Target Name: 204.2.229.9
IP: 204.2.229.9
Date/Time: 1/6/2018 11:46:50 PM - 1/6/2018 11:56:50 PM

Hop Sent PL% Min Max Avg Host Name / [IP]
1 600 0 0.33 79.25 1.71 SYNOLOGYROUTER [192.168.1.1]
2 600 0 3.71 104.06 8.66 10.82.8.1 [10.82.8.1]
3 600 0 3.70 53.78 8.98 ip68-4-11-194.oc.oc.cox.net [68.4.11.194]
4 600 0 4.85 28.11 10.40 ip68-4-11-16.oc.oc.cox.net [68.4.11.16]
5 600 0 7.48 130.17 14.30 68.1.1.63 [68.1.1.63]
6 600 0 7.09 57.89 14.57 ae-7.a00.lsanca20.us.bb.gin.ntt.net [129.250.194.165]
7 600 0 16.19 45.41 23.12 ae-3.r01.lsanca20.us.bb.gin.ntt.net [129.250.2.233]
8 600 0 9.16 31.85 14.87 ae-8.r23.lsanca07.us.bb.gin.ntt.net [129.250.6.48]
9 600 6 31.92 55.99 37.15 ae-12.r22.snjsca04.us.bb.gin.ntt.net [129.250.4.150]
10 600 8 32.19 58.09 38.06 ae-19.r01.snjsca04.us.bb.gin.ntt.net [129.250.3.27]
11 600 10 34.25 66.63 40.83 ae-1.a00.snjsca04.us.bb.gin.ntt.net [129.250.2.228]
12 600 12 35.64 60.32 41.65 xe-0-0-42-1.a00.snjsca04.us.ce.gin.ntt.net [129.250.195.42]
13 600 9 35.58 57.83 40.47 204.2.229.242 [204.2.229.242]
14 600 6 34.38 157.97 41.46 204.2.229.9 [204.2.229.9]

[Bluefve]

Happening to me all day now.

- When the disconnects started occurring: Noticed as of 5 PM PT, still happening.
- Platform (PC, PS4, Mac): PC
- Any error codes or messages you receive upon disconnect (if any): No disconnects, but horrendous lag. Stopping and zipping around.
- Type of connection (wired or wireless): Wireless
- Other programs running while the game is playing (if any): No
- Frequency at which disconnect occurs: Constantly
- Where you are booted to after disconnect (Desktop, Start Screen, Character Select): No disconnection, but noticeable in Quarrymill and in the Palace of the Dead in particular

Target Name: 204.2.229.9
IP: 204.2.229.9
Date/Time: 1/6/2018 11:46:50 PM - 1/6/2018 11:56:50 PM

Hop Sent PL% Min Max Avg Host Name / [IP]
1 600 0 0.33 79.25 1.71 SYNOLOGYROUTER [192.168.1.1]
2 600 0 3.71 104.06 8.66 10.82.8.1 [10.82.8.1]
3 600 0 3.70 53.78 8.98 ip68-4-11-194.oc.oc.cox.net [68.4.11.194]
4 600 0 4.85 28.11 10.40 ip68-4-11-16.oc.oc.cox.net [68.4.11.16]
5 600 0 7.48 130.17 14.30 68.1.1.63 [68.1.1.63]
6 600 0 7.09 57.89 14.57 ae-7.a00.lsanca20.us.bb.gin.ntt.net [129.250.194.165]
7 600 0 16.19 45.41 23.12 ae-3.r01.lsanca20.us.bb.gin.ntt.net [129.250.2.233]
8 600 0 9.16 31.85 14.87 ae-8.r23.lsanca07.us.bb.gin.ntt.net [129.250.6.48]
9 600 6 31.92 55.99 37.15 ae-12.r22.snjsca04.us.bb.gin.ntt.net [129.250.4.150]
10 600 8 32.19 58.09 38.06 ae-19.r01.snjsca04.us.bb.gin.ntt.net [129.250.3.27]
11 600 10 34.25 66.63 40.83 ae-1.a00.snjsca04.us.bb.gin.ntt.net [129.250.2.228]
12 600 12 35.64 60.32 41.65 xe-0-0-42-1.a00.snjsca04.us.ce.gin.ntt.net [129.250.195.42]
13 600 9 35.58 57.83 40.47 204.2.229.242 [204.2.229.242]
14 600 6 34.38 157.97 41.46 204.2.229.9 [204.2.229.9]

[Callinon]

Here's another because shouting in to the wind is fun.

Computing statistics for 400 seconds...
            Source to Here   This Node/Link
Hop  RTT    Lost/Sent = Pct  Lost/Sent = Pct  Address
  0                                           Lapis [192.168.1.54]
                                0/ 100 =  0%   |
  1    0ms     0/ 100 =  0%     0/ 100 =  0%  RT-AC68U-5570 [192.168.1.1]
                                0/ 100 =  0%   |
  2   10ms     0/ 100 =  0%     0/ 100 =  0%  96.120.28.53
                                0/ 100 =  0%   |
  3   10ms     0/ 100 =  0%     0/ 100 =  0%  te-0-7-0-8-sur03.chicago301.il.chicago.comcast.net [68.85.180.45]
                                0/ 100 =  0%   |
  4   10ms     0/ 100 =  0%     0/ 100 =  0%  68.85.183.142
                                0/ 100 =  0%   |
  5   12ms     0/ 100 =  0%     0/ 100 =  0%  be-144-ar01.area4.il.chicago.comcast.net [162.151.45.69]
                                0/ 100 =  0%   |
  6   12ms     0/ 100 =  0%     0/ 100 =  0%  be-33491-cr02.350ecermak.il.ibone.comcast.net [68.86.91.165]
                                0/ 100 =  0%   |
  7   12ms     0/ 100 =  0%     0/ 100 =  0%  be-10577-pe03.350ecermak.il.ibone.comcast.net [68.86.86.2]
                                0/ 100 =  0%   |
  8   14ms     0/ 100 =  0%     0/ 100 =  0%  ae-26.a02.chcgil09.us.bb.gin.ntt.net [129.250.66.65]
                                0/ 100 =  0%   |
  9   12ms     0/ 100 =  0%     0/ 100 =  0%  ae-3.r07.chcgil09.us.bb.gin.ntt.net [129.250.5.111]
                                0/ 100 =  0%   |
 10   16ms     0/ 100 =  0%     0/ 100 =  0%  ae-5.r21.chcgil09.us.bb.gin.ntt.net [129.250.4.213]
                                0/ 100 =  0%   |
 11  131ms     5/ 100 =  5%     5/ 100 =  5%  ae-5.r22.snjsca04.us.bb.gin.ntt.net [129.250.5.17]
                                0/ 100 =  0%   |
 12   81ms     0/ 100 =  0%     0/ 100 =  0%  ae-40.r02.snjsca04.us.bb.gin.ntt.net [129.250.3.121]
                                0/ 100 =  0%   |
 13  180ms    11/ 100 = 11%    11/ 100 = 11%  ae-2.a00.snjsca04.us.bb.gin.ntt.net [129.250.3.58]
                                0/ 100 =  0%   |
 14  180ms     8/ 100 =  8%     8/ 100 =  8%  xe-0-0-42-1.a00.snjsca04.us.ce.gin.ntt.net [129.250.195.42]
                                0/ 100 =  0%   |
 15   77ms     0/ 100 =  0%     0/ 100 =  0%  204.2.229.242
                                2/ 100 =  2%   |
 16  176ms     2/ 100 =  2%     0/ 100 =  0%  204.2.229.97

Trace complete.

[Bluefve]

I've just got off the phone with COX, and its confirmed an issue with ntt.net's infrastructure -- quite possibly, they have a specific route they use for COX customers only, which is the one affected and why it seems only we're getting impacted.

I'd recommend everyone who's with COX call this number and relay your packet loss, as well.

First confirm that it's on their end (CMD trace route or use Ping Plotter to 204.2.229.9 to check results), and not something else.

If that's the problem for you too, do the following:

- Call 1-800-234-3993 (COX Hotline, select tech support in the directory).
- When an agent picks up, state that you're reporting packet loss along a specific route and that you'd like to speak to a Tier II agent. (Tier I can't help you)
- You should get transferred immediately, but might have a long wait time.
- Relay your situation to the Tier II agent, and provide him your evidence. He should understand quickly. He should open a ticket number for you for the investigation, after several questions.

Other ISP customers should do the same thing but with their appropriate hotline.

Edit: Valid steps, but their network team got back -- Still NTT confirmed, but they won't contact directly. Square Enix should reach out to them directly, since they are the direct customer for NTT.

[Callinon]

Hello Callinon Thank you for your reply,

We are going to be monitoring this issue. In the mean time, have you tried contacting your ISP? We suggest you report the issue to them and obtain a reference/ticket number from them. Thank you for your understanding while we try and get this issue resolved.

Thank you for playing Final Fantasy XIV! Hope to see you online.

A ticket number for what? If I call Comcast and tell them "I'm having problems connecting to Final Fantasy XIV" they're going to tell me, quite correctly, that there's nothing wrong with their network. I've shown lots of data on this point. Comcast isn't at fault here. As much as I despise them and wish them ill, they actually aren't the problem here.

[LappelDuvide]

I also have this issue. Cox ISP, in Orange County CA. Usually from 6pm to midnight, I have really terrible packet losses and the occasional disconnect. During the weekends, it can start as early as 3pm. The game is practically unplayable when the packet losses occur. During dungeons and raids, the game goes from playable to a total stand still every 5-15 seconds for 1-10 seconds.

[XenoCranel]

Did they solve your issue or are they still working on it?

This is my tracert

Tracing route to 204.2.229.9 over a maximum of 30 hops

1 <1 ms <1 ms <1 ms 192.168.1.1
2 * * * Request timed out.
3 8 ms 16 ms 12 ms 68.4.15.8
4 29 ms 29 ms 29 ms ip68-4-11-98.oc.oc.cox.net [68.4.11.98]
5 31 ms 30 ms 34 ms 68.1.1.63
6 33 ms 31 ms 34 ms ae-7.a00.lsanca20.us.bb.gin.ntt.net [129.250.194.165]
7 58 ms * 57 ms ae-3.r01.lsanca20.us.bb.gin.ntt.net [129.250.2.233]
8 * 64 ms 70 ms ae-8.r23.lsanca07.us.bb.gin.ntt.net [129.250.6.48]
9 58 ms * 55 ms ae-12.r22.snjsca04.us.bb.gin.ntt.net [129.250.4.150]
10 * 56 ms * ae-19.r01.snjsca04.us.bb.gin.ntt.net [129.250.3.27]
11 44 ms 51 ms 43 ms ae-1.a00.snjsca04.us.bb.gin.ntt.net [129.250.2.228]
12 60 ms 70 ms 61 ms xe-0-0-42-1.a00.snjsca04.us.ce.gin.ntt.net [129.250.195.42]
13 60 ms 59 ms * 204.2.229.242
14 50 ms 42 ms 44 ms 204.2.229.9

UPDATE:

I just got off the phone with a Cox Tier II Agent.

Total time on the phone investigating this issue together, 40 min or so.

In my hop 6-12, the server ntt.net is a japanese owned data center that Cox has no control over.

If you look up via WhoIs for ntt.net, you will find more info.

These are located in the LA area, lsanca means that it is LA (Los Angeles) based.

This is not only a cox issue, this is an issue for players that are being routed via the LA based ntt.net server.

Cox cannot do anything about this, so the issue seems to point toward the FF data center

[drebbe]

Did they solve your issue or are they still working on it?

This is my tracert

Tracing route to 204.2.229.9 over a maximum of 30 hops

1 <1 ms <1 ms <1 ms 192.168.1.1
2 * * * Request timed out.
3 8 ms 16 ms 12 ms 68.4.15.8
4 29 ms 29 ms 29 ms ip68-4-11-98.oc.oc.cox.net [68.4.11.98]
5 31 ms 30 ms 34 ms 68.1.1.63
6 33 ms 31 ms 34 ms ae-7.a00.lsanca20.us.bb.gin.ntt.net [129.250.194.165]
7 58 ms * 57 ms ae-3.r01.lsanca20.us.bb.gin.ntt.net [129.250.2.233]
8 * 64 ms 70 ms ae-8.r23.lsanca07.us.bb.gin.ntt.net [129.250.6.48]
9 58 ms * 55 ms ae-12.r22.snjsca04.us.bb.gin.ntt.net [129.250.4.150]
10 * 56 ms * ae-19.r01.snjsca04.us.bb.gin.ntt.net [129.250.3.27]
11 44 ms 51 ms 43 ms ae-1.a00.snjsca04.us.bb.gin.ntt.net [129.250.2.228]
12 60 ms 70 ms 61 ms xe-0-0-42-1.a00.snjsca04.us.ce.gin.ntt.net [129.250.195.42]
13 60 ms 59 ms * 204.2.229.242
14 50 ms 42 ms 44 ms 204.2.229.9

UPDATE:

I just got off the phone with a Cox Tier II Agent.

Total time on the phone investigating this issue together, 40 min or so.

In my hop 6-12, the server ntt.net is a japanese owned data center that Cox has no control over.

If you look up via WhoIs for ntt.net, you will find more info.

These are located in the LA area, lsanca means that it is LA (Los Angeles) based.

This is not only a cox issue, this is an issue for players that are being routed via the LA based ntt.net server.

Cox cannot do anything about this, so the issue seems to point toward the FF data center

Correct, SE's datacenter is on NTT's servers which is why everyone routes through NTT. There is no fix from our side until SE puts a lot of pressure on NTT or moves companies.