Bad lag again with NTT's NA server network

Yeah today was absolutely horrible for me. Constantly knocked out of lobby with 2002 or 96002 after clicking things and it takes forever to do anything. I hope they fix this soon - making the game very unplayable.

Having issues as well, I have ATT Fiber and was fine until the maintenance and now my ping has increased by 400%, non-stop lag spikes, rubber-banding, and disconnects. Today was especially bad.

Really bad right now too. I have Cox cable internet. Cutting the trace down to where it starts to be problematic, the first timeout:

5 * * * Request timed out.
6 * 18 ms * ash-b2-link.ip.twelve99.net [213.248.76.24]
7 18 ms 17 ms 18 ms ae-40.a08.asbnva02.us.bb.gin.ntt.net [129.250.8.18]
8 18 ms 18 ms 30 ms ae-8.r25.asbnva02.us.bb.gin.ntt.net [129.250.4.22]
9 18 ms 19 ms 19 ms ae-1.a05.asbnva02.us.bb.gin.ntt.net [129.250.3.244]
10 * * * Request timed out.
11 20 ms 26 ms 20 ms ae-0.r24.asbnva02.us.bb.gin.ntt.net [129.250.3.243]
12 91 ms * 88 ms ae-2.r24.snjsca04.us.bb.gin.ntt.net [129.250.6.237]
13 87 ms 87 ms 87 ms ae-4.a00.scrmca03.us.bb.gin.ntt.net [129.250.7.57]
14 88 ms 87 ms 89 ms xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
15 * * * Request timed out.
16 87 ms 87 ms 85 ms 204.2.29.122

Quote:

---

Originally Posted by Kyeria

At this point some of us may not have a choice either. What VPN are you using and do you like it. This issue is getting insanely infuriating...

---

For me, I'm using ExitLag. They're running promotions for the past few months and it's been working relatively well. They do have good customer support.

Having issues for the second night in a row, around the same time (10PM EST). Still having issues as of 11:30PM EST, gave up trying to play for the night.

Been happening to me for the last two nights, usually when the later part of the evening sets in. The ping just jumps up to almost 200, and other players start sliding around like chess pieces on a tilting tray. Tonight there was a really fun thing that I wanted to go to, but it was in Empyreum, and I can't unlock it with the character I wanted to bring because there's no way I can do that specific trial with 150-200 ping. Just no way. I really hope they get this fixed soon. People are paying money to be able to play, and this amount of lag renders the game nigh-unplayable. Even a VPN isn't helping.

Second day running - I've had nothing but LAG and kicked to lobby or game crash. Ran multiple M1-M4 instances on Saturday with no issues, but the last two days clearly have been DDoS or network issues on Square Enix side since nothing on my PC has changed.

Hey servers at at it again! Can we get compensated for the hours we cant play?

https://www.reddit.com/r/ffxiv/s/qYPnJKqzpT

Want to add this Reddit thread of reports last night of similar issues. It doesn't matter the server on NA or the ISP they're using, it's coming back to some routing in NTT's network.

Last few days, it seems the lag issues related to NTT have cropped up again. It tends to start around 10p est and lasts 12+ hours. I can confirm I'm not the only one suffering through this as well, I have multiple friends who I play with that have the exact same issue, and when running a traceroute to the Primal server the issue always happens with NTT -- specifically the San Jose, CA hop. This is making the game absolutely unplayable unless I switch to my phone's hotspot or a VPN where the route changes and doesn't go through NTT's San Jose, CA hop. I shouldn't have to switch to a fucking VPN or a different provider to play a game I'm paying a monthly sub for. Please fix this SE, your community is suffering. I don't care if it's "not your problem", it's affecting YOUR game and the paying customers who play it. Switch providers or something, whatever it takes.

Over the past few days NTT has been making it impossible for me to play.
Sunday: I was being routed through France from NA.

12 34 ms 34 ms 35 ms ae-11.r24.asbnva02.us.bb.gin.ntt.net [129.250.2.138]
13 * 107 ms 106 ms ae-13.r21.parsfr04.fr.bb.gin.ntt.net [129.250.6.5]
14 * * *

Last Night I was booted and couldn't get back into server.

And now this morning it's fine one minute and impossible the next:

15 32 ms 32 ms 31 ms ae-0.r24.asbnva02.us.bb.gin.ntt.net [129.250.3.243]
16 166 ms 163 ms * ae-2.r24.snjsca04.us.bb.gin.ntt.net [129.250.6.237]
17 166 ms 168 ms 167 ms ae-4.a00.scrmca03.us.bb.gin.ntt.net [129.250.7.57]

It's been three days. When I have issues with FFXIV it is always an NTT node.

My game has been fine for the last month but two nights ago around 10 PM EST I get massive packet loss and disconnects.

I'll go to accept a quest, and it takes like 5 seconds for it to complete the action.

Other players sliding around etc.

It clears up after like an hour but it's really annoying as I get home from work at 10PM. Unlucky me.

Day 3 of late night issues. 1:15AM EST start time.

1:25 AM EST issues came back.

Played from 10PM EST to 1:00AM EST without issue.

https://i.imgur.com/jg8O5T5.png

EDIT: Also getting Error 2002 now and can't even connect to the lobby server. I guess i'll take a break.

https://i.imgur.com/vnno0TK.png

Same as above, issues the last 3 nights, just went in to help a M1s party try to clear and game is unplayable, might cancel the sub if this keeps up there's no way to dodge or keep up with anything when the game is like this. Twitch, youtube, discord, etc. everything else is totally fine, only FF having issues. Clueless on what to do about it.

Hear hear.
NY Spectrum here. The past couple days had small spurts of lag. Annoying but not really game affecting.

Today, forget it. Dropped and can't get back in. Using Cloudflare WARP like suggested on this forum and the problem's gone but rather not have to rely on a "VPN" to be able to play decently.

Yeah. I've been having problems the last couple of nights too. Last night I got home from work and tried to do a Frontline and got DC'd while trying to load into it, then lagged off and on throughout it when I got back in. Tonight, I logged in and was able to do a dungeon just fine, but after taking a bit of a break I tried to do another, and found that I was lagging so hard that it took a good few seconds to switch back to the Job that I've been leveling. And doing the second dungeon... Yeah. I didn't even make it to the first boss before I decided that it just wasn't going to happen and dropped out (don't worry, it was a Trust so no penalty and no one was left waiting for a replacement).

This is honestly getting a bit frustrating. My connection before leaving for work was just fine so I might just go back to playing other games when I get home from work and only play FFXIV in the morning if this doesn't clear up soon.

I am curious if I am being routed through a different node and seem to be avoiding this hateful NTT node in San Jose? Florida Spectrum here. Playing on PS5. Dynamis/Seraph. I've experience some slow down and lag here and there this past week. But nothing to the degree folks are experiencing here.

Greetings,

We are aware that some players have been experiencing network-related issues (such as a few seconds of network loss, lag, or disconnects) over the past few days when playing on the North American data centers.

To help us with our efforts in contacting carriers we have relationships with, we would greatly appreciate it if you could submit traceroute data gathered during the instances that these network issues occur.

Please report through the "Providing information on packet loss and lag" section at the bottom of this support form. Instructions on how to obtain your traceroute information can be found in this Lodestone announcement.

Additionally, as mentioned in the announcement, even if we are unable to request investigations from some of the ISPs from our end, the situation may be improved if players can contact their internet service provider and share their details and issues directly with them.

We appreciate your cooperation as we continue striving to address network-related issues that players may be experiencing.

Been having similar issues for the last few days as well. I haven't had any connection issues prior to this aside from when the servers have been DDOSed and gameplay is usually smooth as butter. Currently however, players and NPCs are sliding around, rubber banding, actions - even bringing up retainers - taking a while to register, enemy attacks and AoEs happen all at once as the game tries to catch up, a few disconnects, etc. The game is basically unplayable in any real capacity for me currently.

My connection is rock solid with everything aside from FFXIV. Running the Windows version and in Massachusetts using Xfinity.

Been there, done that. Nobody that's high enough to do anything is in contact with any of the customers. Spectrum's answer was basically a shrug and "iunno man, ain't our issue, sucks for ya tho".

Forgive me if this sounds rude or cross, but hasn't one of the posters in this thread isolated the problem area and identified precisely where the problem is happening -- that is, NTT's San Jose node? I admit I don't know or understand what Squeenix's back-end policies and practices regarding this sort of thing are, but it seems a large waste of time and energy to cast the net wide when the exact area of the problem may have already been determined.

I would like to second the issue being with an NTT San Jose node. I ran multiple traceroutes over the course of an hour, from two different IP addresses (located in the same place), and the issue popped up in the same spot every time. Please see below (with some unrelated nodes removed for privacy):

8 68 ms 54 ms 50 ms ae-25.a06.chcgil09.us.bb.gin.ntt.net [129.250.8.61]
9 40 ms 56 ms 51 ms ae-10.r23.chcgil09.us.bb.gin.ntt.net [129.250.3.37]
10 50 ms 49 ms 49 ms ae-11.r24.asbnva02.us.bb.gin.ntt.net [129.250.2.138]
11 * 92 ms 48 ms ae-0.a05.asbnva02.us.bb.gin.ntt.net [129.250.3.242]
12 * * * Request timed out.
13 * * * Request timed out.
14 326 ms * * ae-2.r24.snjsca04.us.bb.gin.ntt.net [129.250.6.237]
15 297 ms 345 ms 323 ms ae-4.a00.scrmca03.us.bb.gin.ntt.net [129.250.7.57]
16 354 ms 466 ms * xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
17 * * * Request timed out.
18 347 ms 368 ms 316 ms 204.2.29.122

See above the two request timeouts before connecting to the San Jose node snjsca04 129.250.6.237.

The next traceroute I ran, however, came out like this:

8 114 ms 50 ms 73 ms ae-25.a06.chcgil09.us.bb.gin.ntt.net [129.250.8.61]
9 124 ms 44 ms 52 ms ae-10.r23.chcgil09.us.bb.gin.ntt.net [129.250.3.37]
10 64 ms 49 ms 52 ms ae-11.r24.asbnva02.us.bb.gin.ntt.net [129.250.2.138]
11 * 154 ms 55 ms ae-0.a05.asbnva02.us.bb.gin.ntt.net [129.250.3.242]
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 * * * Request timed out.
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.

I continued to experience this for the past hour. Every single time, without fail, traceroute shows the issue beginning on the 12th hop after leaving Ashburn. The issue began around 4 or 5am Eastern time for me, and got progressively worse. Multiple times I was disconnected from the game, with the game closing itself once. I also wasn't able to receive messages in the chat, although strangely enough, my party was able to see my messages. I want to emphasize that my internet is otherwise working fine, and I'm able to be in VoIP calls, browse websites, stream videos, and so on, with 300+ Mbps. I was also able to connect and play just fine on Japan servers just fine while this issue was going on (Elemental/Kujata to be precise).

Multiple people with multiple ISPs in Eastern NA are experiencing this same issue that seems to correlate with a particular San Jose node. While playing with my friend who's in Washington state, he had no such issue. There may be a problem with the way players in Eastern US/Canada are being routed to the server.

EDIT: Speak of the devil, just after submitting this reply, I got disconnected from the server with a 90003 error. Traceroute looks exactly the same as above.

EDIT 2: Tested with a VPN routing me through Seattle. The repeated timeouts never occurred, and it never seems to have attempted to connect to ae-2.r2d.snjsca04 129.250.6.237. Here is the traceroute for that one, note the difference in IP addresses between ae-2.r2d.snjsca04 (the one mentioned in my previous traceroute, repeatedly seen in traceroutes posted by players on at least 4 different sites over the past week), and the instances of snjsca04 here (connected to in this traceroute with minor to no issues in comparison):

1 99 ms 101 ms 99 ms [removed]
2 111 ms 100 ms 101 ms [removed]
3 100 ms 111 ms 106 ms ae-50.a03.sttlwa01.us.bb.gin.ntt.net [128.241.15.49]
4 101 ms 104 ms 115 ms ae-3.r25.sttlwa01.us.bb.gin.ntt.net [129.250.2.206]
5 119 ms 116 ms 134 ms ae-3.r25.snjsca04.us.bb.gin.ntt.net [129.250.3.124]
6 134 ms 119 ms 122 ms ae-1.a03.snjsca04.us.bb.gin.ntt.net [129.250.2.191]
7 * * * Request timed out.
8 121 ms 117 ms 117 ms ae-6.r24.snjsca04.us.bb.gin.ntt.net [129.250.2.158]
9 121 ms 138 ms 120 ms ae-4.a00.scrmca03.us.bb.gin.ntt.net [129.250.7.57]
10 121 ms 120 ms 123 ms xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
11 121 ms * 122 ms 204.2.29.241
12 118 ms 121 ms 123 ms 204.2.29.122

12:51 AM EST Packet Loss and Disconnected Again.

Every god damned night at the same time for the past two weeks, unplayable lag and constant 90k/2k disconnects. Spectrum, NE coast. Internet connection is perfectly fine and this is the only game affected. How long is this crap going to continue?

Quote:

---

Originally Posted by ChewieFlakes

Every god damned night at the same time for the past two weeks, unplayable lag and constant 90k/2k disconnects. Spectrum, NE coast. Internet connection is perfectly fine and this is the only game affected. How long is this crap going to continue?

---

I agree. I've avoided posting and have been sending in trace route to the link provided but nothing seems to be happening. This is utterly ridiculous and contacting our ISP does nothing, they just give us the runaround and excuses.

Confirmed, it is indeed Happening Again. Several people have identified the exact problem, and from what it looks like -- though I admit I'm not experienced with such matters, so I have no clue if there's additional concerns that I'm not aware of -- these people have also provided Square-Enix with literally the exact location of the issue. They have the "who", "what", "when", "where", AND "why". So I'm confused as to why this continues, and why Square-Enix seems to be sitting on all the information they need rather than taking action. Again, though, I don't know if there are more considerations that I am unaware of.

A Canadian friend in Ontario, and I in eastern PA are both seeing issues for the past week, every evening around 12:30-1AM we're getting the disconnection errors. This has been really a sour note on our evening grind sessions and I hope someone at SE can assist with getting this fixed sooner rather than later.

This has been going on since Monday/Tuesday Night this week.
I have run the traceroute and sent in an error report to Square.
I have Spectrum in OH. Not my net, it's 940mbps down/41 up. Everything else works, no lag elsewhere, I host a server on MC too and no lag for my friend in Portugal.

I'm not getting bounced to San Jose with the traceroute, but getting bounced from Ohio to ASBNVA to PARIS FREAKING FRANCE before it decides to request time out the rest of the route.

By the twelve, I just wanted to do roulettes tonight. I don't want to mess with a VPN.

Had this happen for the first time too... east coast.

Ive been having the same issue all week. PS5 NA Eastern Time zone. Spectrum. Never had an issue before with this game. This week has been the first and I started back just before Endwalker launched.

Quote:

---

Originally Posted by ChewieFlakes

Every god damned night at the same time for the past two weeks, unplayable lag and constant 90k/2k disconnects. Spectrum, NE coast. Internet connection is perfectly fine and this is the only game affected. How long is this crap going to continue?

---

That really sucks. Especially when your on standby in game, not sure if the lag will persist or go. The worst day for me was Wednesday. Everyone was rubber banding around me for about an hour.

The route from Hong Kong to US is also unusual and has high latency, as SE is currently routed through Singapore, then Osaka, and finally Sacramento:

traceroute to 204.2.29.99 (204.2.29.99), 30 hops max, 32 byte packets
1 *
2 10.20.1.0 0.51 ms * LAN Address
3 *
4 HundredGE0-3-0-0.br02.hkg08.as3491.net (63.223.29.194) 1.48 ms AS3491,AS31713 China, Hong Kong, pccw.com
5 63.218.205.154 1.71 ms AS3491,AS31713 China, Hong Kong, pccw.com
6 129.250.5.166 2.10 ms AS2914 China, Hong Kong, ntt.com
7 *
8 ae-0.a03.sngpsi07.sg.bb.gin.ntt.net (129.250.5.238) 30.84 ms AS2914 Singapore, ntt.com
9 *
10 *
11 ae-4.r27.osakjp02.jp.bb.gin.ntt.net (129.250.2.67) 80.30 ms AS2914 Japan, Osaka, ntt.com
12 ae-0.r26.osakjp02.jp.bb.gin.ntt.net (129.250.3.44) 82.21 ms AS2914 Japan, Osaka, ntt.com
13 ae-4.r24.snjsca04.us.bb.gin.ntt.net (129.250.2.118) 191.99 ms AS2914 United States, California, San Jose, ntt.com
14 ae-4.a00.scrmca03.us.bb.gin.ntt.net (129.250.7.57) 187.92 ms AS2914 United States, California, Sacramento, ntt.com
15 xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net (128.241.2.18) 188.15 ms AS2914 United States, California, Sacramento, ntt.com
16 204.2.29.241 187.69 ms AS2914 United States, California, Sacramento, ntt.com
17 204.2.29.99 187.64 ms AS2914 United States, California, Sacramento, ntt.com

I understand why SE is using NTT IP Transit since it's one of the most cost effective and tier 1 DDoS mitigation services available, but other service providers with NTT IP Transit are not routed through Singapore, here using xTOM hosting IP as an example:

traceroute to 104.245.12.12 (104.245.12.12), 30 hops max, 32 byte packets
1 *
2 10.20.1.0 0.42 ms * LAN Address
3 63.222.114.16 0.65 ms AS3491,AS31713 China, Hong Kong, pccw.com
4 bundle-ether41.br02.hkg08.as3491.net (63.218.204.154) 1.74 ms AS3491,AS31713 China, Hong Kong, pccw.com
5 bundle-ether41.br02.hkg08.as3491.net (63.218.204.154) 1.41 ms AS3491,AS31713 China, Hong Kong, pccw.com
6 63.218.205.154 1.47 ms AS3491,AS31713 China, Hong Kong, pccw.com
7 ae-16.r26.tkokhk01.hk.bb.gin.ntt.net (129.250.5.164) 14.51 ms AS2914 China, Hong Kong, ntt.com
8 ae-10.r32.tokyjp05.jp.bb.gin.ntt.net (129.250.2.50) 49.04 ms AS2914 Japan, Tokyo, ntt.com
9 *
10 ae-1.a03.snjsca04.us.bb.gin.ntt.net (129.250.2.191) 156.84 ms AS2914 United States, California, San Jose, ntt.com
11 ntt.sjc.xtom.us (157.238.226.245) 158.28 ms AS2914 United States, California, San Jose, ntt.com
12 cs03.q51.sjc.xtom.us (91.200.241.87) 160.43 ms * United States, California, San Jose, xtom.com
13 sjc.speedtest.vps.hosting (104.245.12.12) 156.83 ms AS6233 United States, California, San Jose, xtom.com

This suggests that there must be some aggressive load balancing causing these unusual routing patterns. Since I know my service provider has direct peering with NTT, there is nothing can be done from our end.

-----------------------------

At this point of time, the route seems to have changed and now looks perfectly normal again:

traceroute to 204.2.29.99 (204.2.29.99), 30 hops max, 32 byte packets
1 *
2 10.20.1.0 0.50 ms * LAN Address
3 63.222.114.16 0.71 ms AS3491,AS31713 China, Hong Kong, pccw.com
4 HundredGE0-3-0-0.br02.hkg08.as3491.net (63.223.29.194) 1.37 ms AS3491,AS31713 China, Hong Kong, pccw.com
5 63.218.205.154 1.46 ms AS3491,AS31713 China, Hong Kong, pccw.com
6 *
7 ae-16.r26.tkokhk01.hk.bb.gin.ntt.net (129.250.5.164) 2.46 ms AS2914 China, Hong Kong, ntt.com
8 ae-10.r32.tokyjp05.jp.bb.gin.ntt.net (129.250.2.50) 51.85 ms AS2914 Japan, Tokyo, ntt.com
9 *
10 *
11 ae-4.a00.scrmca03.us.bb.gin.ntt.net (129.250.7.57) 164.54 ms AS2914 United States, California, Sacramento, ntt.com
12 xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net (128.241.2.18) 164.09 ms AS2914 United States, California, Sacramento, ntt.com
13 204.2.29.241 163.63 ms AS2914 United States, California, Sacramento, ntt.com
14 204.2.29.99 163.31 ms AS2914 United States, California, Sacramento, ntt.com

Ope, there it goes again. 10:30-ish AM CST, Spectrum customer. Ping shot up to 135 and it was taking 5-10 seconds for the game to even register that I was trying to exit, and I had to do it twice before it actually decided to work.

Have been experiencing the same thing. NE USA, Spectrum. Before, I was only experiencing it past around 9 pm EST, but now I've just logged in and it's impossible to do anything in game again. It took a solid two minutes for my 'log out' button click to register. Really disappointed that Square Enix seems to not care that we cannot play the product we paid for.

I just registered my account to the forum so I could post this, but I'm experiencing the issue at a novel time of day now. I don't have a traceroute to post because I don't know the next thing about how to do that. For me the lag has always started sometime between midnight and 02:00 AM in my time zone (Eastern US), but about 15 minutes ago it began over 12 hours ahead of that range. This is the first time over the past week that I have personally had this happen in the middle of the day, so I'm praying it isn't indicative of the situation actively worsening. I'm frustrated of course, but I'm not so frustrated yet that I'd get a separate subscription to have a VPN only so I can actually use my XIV subscription. Here's hoping it doesn't persist, and lord willing that Square Enix gets back to us about this more substantially.

Unsubbed because this constant lag makes the game unplayable. There's no other solution from the user's end. Get your shit together.

In an earlier post someone talked about a CloudFlare app that worked for them if you didn't want to have to buy a VPN subscription. Unfortunately, of late I've found that this solution no longer works, and the only way to deal with it is to wait for the lag to work itself out, so I'm not sure that anyone else will have luck with it. But in case it does, here's the link again:

https://1.1.1.1/

Hopefully it will help some of you avoid the lag. It no longer works for me.

It was nice to have one day (Almost two days) of no issues, but starting last night disconnects and LAG started all over again. Appears again today (Friday) to be in full swing again. So glad I ran all my raids yesterday before it got bad.

Is there any plans to resolve this long term at all? I came back to try the new expansion after beating base Endwalker, but I can't even get to the new expansion because of technical issues that appear as if they've been ongoing for over a year now If I'm understanding this right. I just can't see sticking around If that's the case. It seems odd that something like this could occur for that long from people who went to the extreme of preventing purchase of the game for server stability.

Absolutely wild that this is (STILL) a thing. I've been locked out of the game for an entire day now, my connection never makes it through. At first tracert was pointing the finger at the Sacramento node, but now NTT is trying to route me through Paris, France?? Despite how frequently this occurs and how many people it must affect, I'm surprised Square hasn't said a word about it.