Bad lag again with NTT's NA server network

Yep, the entire NA datacenter's down. Probably not related to the AT&T issue, but who knows if that'll improve things.

Maybe the hard restart will help... *largest whiff of copium*

+1 AT&T Fiber customer, Southeast, over here as well, just sent the support form in with the traceroute. Been rubber banding all week. Some PVE 4/8 man content has been doable with some deaths, which is crappy if I'm the healer. PVP is absolutely a no go.

Have tested a couple VPNs, they lower the ping to "acceptable" levels I suppose, but I'm used to much better than this.

Yep, same Arimaya.

As for the "copium" bit, sometimes it takes things breaking completely in order to find the fix.

Servers going down solved my issues, too bad I cant play to enjoy it

https://cdn.discordapp.com/attachmen...2546/image.png

Quote:

---

Originally Posted by TaleraRistain

Right at the time the lag issues normally clear up, too.

---

Yeah, perfect timing. ._.;

Note that this issue was always resolving around 12am PST this past week, and then picking up again sometime around noon.

Just a follow-up at least for AT&T South-East region lag issues are gone for me. No packet loss after the maintenance completion. TY SE for the prompt response especially considering the patch is around the corner. ^_^ I hope everyone has been similarly fixed.

I'm skeptical, but I'll keep my tracer and ping tracker running today.

From what I see it was only the lobby server that stopped responding for a very short time. There was even a lodestone post about that.

https://na.finalfantasyxiv.com/lodes...ef015671754060

Lobby server stopped responding between 2:52 and 3:04am EST. Monitored world server didn't.
Packet loss also literally stopped at 2am EST like clockwork.
https://imgur.com/KiFponm.jpg

I'm not getting my hopes up though.

Quote:

---

Originally Posted by MoonmoonMoonmoon

I'm skeptical, but I'll keep my tracer and ping tracker running today.

---

Also skeptical. Still got a couple more hours until that 12-hour cycle starts again.

FWIW, in my haste to get *something* working in readiness for 6.4, I signed up for a different ISP, Xfinity, and put my PS5 on that service last night (plus a laptop to run traceroute/ping)

On Xfinity, pings/latency stayed around 70ms and no lag or rubberbanding.

I did a couple traceroutes at the same time, one going out of ATT Fiber and one going out Xfinity but same physical location / house.

They both went through some of the nodes we've been seeing, like going through the Chicago, San Jose and Sacramento NTT nodes and included IPs like 128.241.2.18 and 129.250.4.239 but Xfinity never went through 129.250.7.57.

The ATT connection had 20% packet loss and 120ms pings spiking to 200ms but the Xfinity connection was perfectly fine.

I cancelled my sub because of this, but have a few days left of play time. I'm also skeptical about last nights crash/reset/outage being the permanent fix, but it seems to be working fine for me right now. Time will only tell.

I saw on the lodestone that this issue was fixed! Can anyone confirm that they aren't having issues around prime time?

Thank you!

Quote:

---

Originally Posted by Kwehvehpeh

I saw on the lodestone that this issue was fixed! Can anyone confirm that they aren't having issues around prime time?

Thank you!

---

Sadly I think the Lodestone post was just about an outage that occurred last night--none of the posts I'm seeing on there look like they pertain to our exact issue. I'm going to be keeping an eye on ping and packet loss today and will see how things look around when the issue normally starts to rear its head.
I'm not holding out hope, though. At this rate I'm probably just going to cancel my sub until we get an actual acknowledgement that this specific issue has been resolved.

Quote:

---

Originally Posted by Xiozen

TY SE for the prompt response especially considering the patch is around the corner. ^_^ I hope everyone has been similarly fixed.

---

Not to be a downer, but prompt isn't the word I'd use.

I'm very skeptical that the issue has actually been resolved. But I hope it has. Time will tell, it's usually around now that it starts to act up. I'll run my own tracert in a few hours and report back.

Quote:

---

Originally Posted by Illyrian94

Sadly I think the Lodestone post was just about an outage that occurred last night--none of the posts I'm seeing on there look like they pertain to our exact issue. I'm going to be keeping an eye on ping and packet loss today and will see how things look around when the issue normally starts to rear its head.
I'm not holding out hope, though. At this rate I'm probably just going to cancel my sub until we get an actual acknowledgement that this specific issue has been resolved.

---

Ah, that's a shame then. Despite knowing better, I'm still holding out hope!

Quote:

---

Originally Posted by Kwehvehpeh

Ah, that's a shame then. Despite knowing better, I'm still holding out hope!

---

The hopium/copium train is that the server outage was due to NTT fixing their node / doing maintenance, which interrupted all FF servers (since NTT is what's used to get there, it's just some unfortunate folks (us) go through a bad node). That's the train I'd like to be on... I'm tentatively on it. One toe in.

Quote:

---

Originally Posted by Lahtori

The hopium/copium train is that the server outage was due to NTT fixing their node / doing maintenance, which interrupted all FF servers (since NTT is what's used to get there, it's just some unfortunate folks (us) go through a bad node). That's the train I'd like to be on... I'm tentatively on it. One toe in.

---

Yeah. I've been holding out on getting a VPN, hoping that they would get it resolved before the weekend. But if its still bad today, I suppose I'll dish out the money for it again.
I pray that we do not have to return to the lagging sands tonight.

Quote:

---

Originally Posted by Kwehvehpeh

Yeah. I've been holding out on getting a VPN, hoping that they would get it resolved before the weekend. But if its still bad today, I suppose I'll dish out the money for it again.
I pray that we do not have to return to the lagging sands tonight.

---

I didn't try playing the game on the VPN, but I downloaded Cloudfare Warp and gave a tracert last night, which showed normal-ish response times (50-70).

As of 10:21am CST, my tracert (without VPN) is normal. I'll check again later.

Quote:

---

Originally Posted by Lahtori

I didn't try playing the game on the VPN, but I downloaded Cloudfare Warp and gave a tracert last night, which showed normal-ish response times (50-70).

As of 10:21am CST, my tracert (without VPN) is normal. I'll check again later.

---

Many thanks, friend! I look forward to seeing updates!

Just did some quick ping commands while Cloudflare WARP was Disabled, at 10:40 AM Central.

First one is to the main NA Datacenter Lobby, the second is to the Aether Datacenter:

Pinging 204.2.29.122 with 32 bytes of data:
Reply from 204.2.29.122: bytes=32 time=63ms TTL=49
Reply from 204.2.29.122: bytes=32 time=73ms TTL=49
Reply from 204.2.29.122: bytes=32 time=58ms TTL=49
Reply from 204.2.29.122: bytes=32 time=59ms TTL=49

Ping statistics for 204.2.29.122:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 58ms, Maximum = 73ms, Average = 63ms


Pinging 204.2.29.6 with 32 bytes of data:
Reply from 204.2.29.6: bytes=32 time=64ms TTL=50
Reply from 204.2.29.6: bytes=32 time=64ms TTL=50
Reply from 204.2.29.6: bytes=32 time=62ms TTL=50
Reply from 204.2.29.6: bytes=32 time=58ms TTL=50

Ping statistics for 204.2.29.6:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 58ms, Maximum = 64ms, Average = 62ms


With Cloudflare WARP Enabled, these are usually both around 70ms, so still acceptable.

So far looking alright for me as well (Indiana, ATT fiber). Every other day this week when I checked over my lunch ping values were in the 100ms range w/ lots of packet loss.

https://i.imgur.com/b16Ubee.png

Fingers crossed!

Quote:

---

Originally Posted by Krojack

From what I see it was only the lobby server that stopped responding for a very short time. There was even a lodestone post about that.

https://na.finalfantasyxiv.com/lodes...ef015671754060

Lobby server stopped responding between 2:52 and 3:04am EST. Monitored world server didn't.
Packet loss also literally stopped at 2am EST like clockwork.
https://imgur.com/KiFponm.jpg

I'm not getting my hopes up though.

---

If they can keep it stable going forward I'll celebrate. This "fixed for a day" or "fixed for a week" is getting old

Don't celebrate just yet, folks.

Tracing route to 204.2.29.122 over a maximum of 30 hops


8 29 ms 30 ms 26 ms ae-5.a00.dllstx14.us.bb.gin.ntt.net [129.250.8.237]
9 27 ms 27 ms 33 ms ae-2.r20.dllstx14.us.bb.gin.ntt.net [129.250.4.9]
10 81 ms 80 ms 82 ms ae-2.r24.lsanca07.us.bb.gin.ntt.net [129.250.7.69]
11 100 ms 117 ms 96 ms ae-5.a01.scrmca03.us.bb.gin.ntt.net [129.250.2.7]
12 112 ms 108 ms 104 ms ae-1.a00.scrmca03.us.bb.gin.ntt.net [129.250.4.76]
13 109 ms 103 ms 110 ms xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
14 * 72 ms * 204.2.29.241
15 97 ms 97 ms 106 ms 204.2.29.122

I lost power at my place for about 2 1/2 hours so I don't have ping graphs for when the lag starts.

Power back and graphs are graphing. Looks like the pings are still bad. It's hovering from 80-100ms. Like in the past, I don't expect packet loss to start till 1 or 2pm. So in about an hour we will see.

But so far not looking good.

https://imgur.com/zCc50TB.jpg

Dang, and I was hoping the servers taking a dump last night would get rid of this garbage.

Alas, back to our posts we go. Remain vigilant, and keep hounding AT&T, NTT, and Square Enix. Pressure and social media callouts work.

Looks like I spoke too soon. NTT nodes spiked to 100+ms of latency and packet loss has started.
Kinda doubt they'll get this fixed by 6.4 at this rate.

https://i.imgur.com/PWtMGvU.png

They'd have to act quickly before maintenance if they want to save face.
That, or explicitly take time during server maintenance to get with NTT and AT&T to fix and test things.

One has to hope a proper Lodestone post about this serious problem would come if it persists into 6.4.

Quote:

---

Originally Posted by Illyrian94

Looks like I spoke too soon. NTT nodes spiked to 100+ms of latency and packet loss has started.
Kinda doubt they'll get this fixed by 6.4 at this rate.

https://i.imgur.com/PWtMGvU.png

---

That is really heartbreaking to see.

Hmm... Mine are still okay. There's two nodes that jump to 90ms, but at least it's not all nodes.

https://i.imgur.com/hbxMpP7.png

Quote:

---

Originally Posted by Lahtori

Hmm... Mine are still okay. There's two nodes that jump to 90ms, but at least it's not all nodes.

https://i.imgur.com/hbxMpP7.png

---

Hmm, an interesting discrepancy from hops 10 and on. The IP address ending in 18 has the lowest ms of all the Sacramento nodes. How curious.

It's back in style. The hopium train has derailed before leaving the station.

https://i.imgur.com/P0wtORf.png

Right on schedule, it's back on the BS

5 9 ms 12 ms 13 ms 32.130.17.75
6 8 ms 16 ms 13 ms cgcil402igs.ip.att.net [12.122.133.161]
7 8 ms 9 ms 8 ms 192.205.32.194
8 8 ms 8 ms 9 ms ae-4.r23.chcgil09.us.bb.gin.ntt.net [129.250.4.239]
9 * 55 ms 55 ms ae-1.r24.snjsca04.us.bb.gin.ntt.net [129.250.5.17]
10 111 ms 110 ms 107 ms ae-4.a00.scrmca03.us.bb.gin.ntt.net [129.250.7.57]
11 111 ms 108 ms 100 ms xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
12 * * * Request timed out.
13 109 ms 109 ms 110 ms 204.2.29.6

I use AT&T fiber, and same thing happens here, lag and packet loss between 12:00pm to 12:00am EST.
Have sent the report. Hopefully they'll get it fixed before patch 6.4 release.

8 35 ms 43 ms 29 ms ae-8.r25.atlnga05.us.bb.gin.ntt.net [129.250.4.66]
9 38 ms 36 ms 37 ms ae-6.r21.dllstx14.us.bb.gin.ntt.net [129.250.4.116]
10 38 ms 37 ms * ae-1.r20.dllstx14.us.bb.gin.ntt.net [129.250.3.54]
11 110 ms 114 ms 113 ms ae-2.r24.lsanca07.us.bb.gin.ntt.net [129.250.7.69]
12 120 ms 122 ms 122 ms ae-5.a01.scrmca03.us.bb.gin.ntt.net [129.250.2.7]
13 114 ms 121 ms 118 ms ae-1.a00.scrmca03.us.bb.gin.ntt.net [129.250.4.76]
14 123 ms 122 ms 121 ms xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
15 * * * Request timed out.
16 120 ms 118 ms 121 ms 204.2.29.8

Hopefully. Thanks for your report, HoneyLamp.
They do help keep the pressure on all parties involved to get things done.

But I will admit, five days of this is more than unacceptable. It's abysmal, and a long list of words that I can't say in public.

I just tweeted FFXIV again, and this time gave NTT a tweet too. Honestly, if it's not resolved by patch day, it may be time to look at the dreaded "game news". I refuse to believe there is no power SE cannot use to push NTT into action - it's their nodes.

Also got my partner to submit his own report to SE.

I'm just at a loss of words. Five days and on going.

Between this and being affected by the sound issue during 6.0 where I couldn't play with sound or it was horrendously garbled, it's definitely been a rough few months for me with this game. Worst I've experienced in 9 years of playing. I'm not angry, just disappointed. I know this isn't in SE's direct control and hope they can leverage some influence. Or maybe move to a different option if that's possible if the issue is with NTT and NTT isn't going to give US customers with certain ISPs good service.

Honestly, I hope they reconsider their deal with NTT in North America. Or at least, SE and/or AT&T should give NTT an ultimatum due to these problems.
Either fix them immediately or find another (more competent) partner for these nodes.

Quote:

---

Originally Posted by Janyd

Honestly, I hope they reconsider their deal with NTT in North America. Or at least, SE and/or AT&T should give NTT an ultimatum due to these problems.
Either fix them immediately or find another (more competent) partner for these nodes.

---

Yeah I was talking to my husband about it last night. He read a book called Tokyo Vice and I guess the gentlemen in it described that sometimes these things in Japan are monopolies in everything but name only. Everyone agrees that X will handle this or Y will handle this. And of course when you have a setup like that, the offender has no impetus to change. Their US servers were in another place in the past, though, so if there is another option then I would hope they'd at least look into it.

Its fascinating the differences between two traceroutes from the same physical location but two different ISPs -- yet still going through similar NTT nodes once it gets to California:

ATT:

Quote:

---

7 cgcil402igs.ip.att.net (12.122.132.197) 8.477 ms 8.791 ms 8.019 ms
8 192.205.32.194 (192.205.32.194) 15.443 ms 7.090 ms 20.245 ms
9 ae-4.r23.chcgil09.us.bb.gin.ntt.net (129.250.4.239) 7.508 ms 7.972 ms 8.069 ms
10 * ae-1.r24.snjsca04.us.bb.gin.ntt.net (129.250.5.17) 55.329 ms 60.330 ms
11 ae-4.a00.scrmca03.us.bb.gin.ntt.net (129.250.7.57) 111.620 ms 113.681 ms 112.283 ms
12 xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net (128.241.2.18) 113.674 ms 113.522 ms 114.396 ms
13 * * 204.2.29.241 (204.2.29.241) 114.096 ms

---

Comcast:

Quote:

---

9 be-2311-pe11.350ecermak.il.ibone.comcast.net (96.110.33.202) 19.727 ms
10 ae-3.a06.chcgil09.us.bb.gin.ntt.net (129.250.8.65) 29.949 ms 23.228 ms 21.926 ms
11 ae-10.r23.chcgil09.us.bb.gin.ntt.net (129.250.3.37) 45.485 ms 26.543 ms 21.283 ms
12 ae-1.r24.snjsca04.us.bb.gin.ntt.net (129.250.5.17) 65.812 ms 74.105 ms 68.419 ms
13 ae-4.a00.scrmca03.us.bb.gin.ntt.net (129.250.7.57) 70.537 ms 71.024 ms 74.866 ms
14 xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net (128.241.2.18) 67.851 ms 73.041 ms 70.355 ms

---

Something is definitely going on between NTT + ATT - who knows whose fault it is at this point but good grief can't someone update their routing tables to get around their own congestion? *sigh*