Bad lag again with NTT's NA server network

I dunno there's also a post on the ATT forums, unrelated to FFXIV, where OP's company uses NTT's services and is also being told it's a peering issue between ATT and NTT.

Also from what I saw on the ATT Forums; Please make sure you ID this as a peering issue for sure. There's this thread over there: https://forums.att.com/conversations...f8a752501d13bc . That guy is having trouble and said so with the NTT backbone, might not be related to FFXIV but I'd recommend directing your howls there, otherwise you may keep getting canned responses. MAKE SURE YOU SAY PEERING ISSUE. That's the magic words.

[edit] Jynx! Guess you saw the same post, Ruined-Childhood.

It's good to see that there are other services than just FFXIV that are being affected by this issue. The post on the AT&T forum, however, is still only a single post from an affected user, and does not include any technical data. At best they claim they were "told" it was a "peering issue between AT&T and the backbone of (their) ISP provider which uses NTT"

Even in this thread, people have claimed that NTT have said that it's AT&Ts fault, but the data simply doesn't line up with what we can see. To reiterate, as far as our data provides, the issue occurs on the inter/intraconnection between the Dallas and Los Angeles NTT DCs. This is technically a "peering" connection, and if this connection is operated by AT&T, it could lead some credence to the allegation by NTT that it is an AT&T issue, however as I have said in my last post, if the connection between the Dallas and Los Angeles was run solely by AT&T, it would affect all customers both up and downstream.

As we continue to have no word from a reputable source (and even the AT&T response in that thread is simply "we think the peering team might best assist you based on what you described), the actual data that we have continues to indicate NTT as the culprit in these issues. I'm not saying that contacting AT&T is fruitless, nor that we shouldn't attempt to spread the word on this far and wide, but if we take the pressure off of NTT (which is what NTT wants us to do, based on their alleged responses), and if it IS an NTT issue, as suspected, then it will let NTT continue to get away with it while people chase red herrings anywhere else.

I don't believe that this is as simple as saying "It's Company Y's fault". I do believe that there are many hands in the pot which has fallen over here. However, with the data we have, the largest hand in that pot is NTT, and as such, they should be the ones who at least make a public, technical announcement which at least acknowledges the problem, and preferably identifies the path towards a solution. If it's the inter/intraconnection between their DCs and that peering is operated by AT&T through a direct contact with AT&T, then sure, but that still wouldn't explain why this became an issue overnight on such a regular, exact, and predictable schedule.

Something smells fishy with this whole issue, and I doubt we'll ever know everything.

Here's hoping for the best. Much love.

I was not expecting other people to be running into NTT issues. If this is impacting people on an even larger scale than just XIV users because of NTT, a fire might be lit and that's good. Absolutely terrible experience and garbage how long this has been going on.

As of right now, I'm seeing significantly different behavior than the "normal" we've had over the past 10+ days. Ping is bouncing around some, but it isn't pegged above 100ms and so far no packet loss. Going to leave Ping Plotter running while I work this afternoon and see if/when things start to get dicey.

https://i.imgur.com/SrcIz0E.png

Same crap new day for me. It does seems to be getting off to a slower start though. Maybe other businesses that host with NTT are moving their services to other hosting providers?

https://i.imgur.com/9DAruME.jpg

Reading the answers from AT&T on their forums is discouraging at best. Over and over "send traceroutes" or "you can track who the node belongs to". We get it AT&T, it's an NTT node. But it's *their* customers who are affected. They seem to just want to pass the buck instead of getting answers too.

Anyone ever find out how we can contact NTT directly?

Just checked after a meeting and now we're back to having ping pegged around 100ms+ and packet loss. Starting later in the day (for me at least) but definitely still broken.

https://i.imgur.com/r4MIRhq.png

Quote:

---

Originally Posted by Rekts

Reading the answers from AT&T on their forums is discouraging at best. Over and over "send traceroutes" or "you can track who the node belongs to". We get it AT&T, it's an NTT node. But it's *their* customers who are affected. They seem to just want to pass the buck instead of getting answers too.

Anyone ever find out how we can contact NTT directly?

---

Some people have contacted NTT and that's how we found it was a peer problem between ATT and NTT but NTT said they can't say more because of an NDA. Most likely AT&T made them sign this with the peer connection.

https://i.imgur.com/OP8NQav.jpg

I think that says it all. I ain't playing this afternoon. Not on Uverse.

Also checked with my phone. NOPE. No go with that either /sigh. Guess I'm playing in the morning or doing island until this is fixed; ugh.

I have been experiencing some rubber-banding since 6.3, at first i though it was my graphics but i played at min specs and the issue persisted, then i though it was my ISP but then i can play other MMOS like wow and guild wars without rubber banding not to mention other online games. this Seems to be a specific issue with FF14 it gets really bad in alliance raids, on dungeons is noticeable and even in trust is present. i have re-installed multiple times and even updated my graphics card. i been keeping an eye on the send and receive(wifi icon in game) and its all over the place it goes from 20 to 300 to 900 in a matter of second and it coincided with the rubber-banding. has anyone found a workaround or a fix to this problem or this is an ongoing issue?

Quote:

---

Originally Posted by Plaides_Nova

I have been experiencing some rubber-banding since 6.3, at first i though it was my graphics but i played at min specs and the issue persisted, then i though it was my ISP but then i can play other MMOS like wow and guild wars without rubber banding not to mention other online games. this Seems to be a specific issue with FF14 it gets really bad in alliance raids, on dungeons is noticeable and even in trust is present. i have re-installed multiple times and even updated my graphics card. i been keeping an eye on the send and receive(wifi icon in game) and its all over the place it goes from 20 to 300 to 900 in a matter of second and it coincided with the rubber-banding. has anyone found a workaround or a fix to this problem or this is an ongoing issue?

---

This is an ongoing issue for folks with AT&T internet (and some folks with other ISPs as well)--your best bet is to use something like Cloudflare Warp or a VPN service as these are workarounds that the community has figured out.

This is confusing ngl. Either it's an NTT node problem or an ATT peer point issue. But neither of them will give a clear answer. I suppose it's not conceivable for SE to just reroute away from the problem area is it?

As expected, AT&T's response is to point a finger back at Square and NTT and say we need to reach out to them.

https://forums.att.com/conversations...05de56a01bcff0

I'm going to open yet another support ticket with SE. Not that I expect anything to come of it, but I figure I can at least be a continual support issue for them if they're going to continue to refuse to acknowledge the issue and bounce us back and forth to AT&T. What a joke of a situation.

Quote:

---

Originally Posted by Illyrian94

As expected, AT&T's response is to point back at Square and NTT and say we need to reach out to them.

https://forums.att.com/conversations...05de56a01bcff0

I'm going to open yet another support ticket with SE. Not that I expect anything to come of it, but I figure I can at least be a continual support issue for them if they're going to continue to refuse to acknowledge the issue and bounce us back and forth to AT&T. What a joke of a situation.

---

From the response of the "expert"(not even an AT&T employee), it's an issue between both NTT and ATT. Everyone really does suck here.

I did a traceroute, and found NTT screwing up. Googled NTT outage issues and found this thread. Glad to see I'm not alone seeing the network issues with FF14. Not glad to see people are having issues. Hopefully NTT can get this resolved :P

We're still in it, huh. I've got about a week and a half left on my sub, and I don't think this will be resolved by the time it expires. Ah, well.

My current trace route (and for reference, I'm up on Lake Erie):

Code:

---

  5    *        *        *    Request timed out.
  6    36 ms    48 ms    12 ms  32.130.17.75
  7    14 ms    15 ms    14 ms  cgcil402igs.ip.att.net [12.122.133.161]
  8    13 ms    18 ms    14 ms  192.205.32.194
  9    14 ms    14 ms    17 ms  ae-4.r23.chcgil09.us.bb.gin.ntt.net [129.250.4.239]
 10    *      64 ms    70 ms  ae-1.r24.snjsca04.us.bb.gin.ntt.net [129.250.5.17]
 11  121 ms  119 ms  120 ms  ae-4.a00.scrmca03.us.bb.gin.ntt.net [129.250.7.57]
 12    *      120 ms  120 ms  xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
 13  119 ms  123 ms    *    204.2.29.241
 14  119 ms  120 ms    *    204.2.29.6
 15  119 ms    *      119 ms  204.2.29.6

---

My ping is better connecting to the European data centers than it is the NA ones. Yikes.

Guys. Just use Mudfish. It's extremely cheap and there are multiple nodes to choose from so if one doesn't work out just pick another one.

I know it's far more unintuitive than other VPNs though.

Quote:

---

Originally Posted by GaiusDrakon

Guys. Just use Mudfish. It's extremely cheap and there are multiple nodes to choose from so if one doesn't work out just pick another one.

I know it's far more unintuitive than other VPNs though.

---

Not everyone can use a VPN sadly. Console doesn’t let you add one and AT&T blocks all of the work arounds that would potentially allow you to use one on your PS. Some people might also not have the extra funds for a second sub to something they shouldn’t even need to be able to play in the first place.

Definitely if you can use a VPN I encourage it and Im happy for everyone who has that option, but at the end of the day what we need is for something to be done and to not allow this problem to get swept under the rug.

Quote:

---

Originally Posted by GaiusDrakon

Guys. Just use Mudfish. It's extremely cheap and there are multiple nodes to choose from so if one doesn't work out just pick another one.

I know it's far more unintuitive than other VPNs though.

---

It's already been pointed out, but yes console players can't use a VPN. Regardless PC players shouldn't have to use a VPN either.

ATT pretty much said definitively that it's an NTT issue on one of the threads of their forum(the problem node is outside their network). If that's actually true, we're not NTT's customer. SE is, they're the ones who will have to pressure NTT to fix it.

And since we still hadn't had any real communication outside of "call your ISP" I don't think we're looking at a resolution anytime soon.

Yeah I'm using a VPN but we shouldn't have to use a VPN to play a game we already pay a sub for.

I check every day before I turn the VPN on and it's still consistently high latency and packet loss on the same schedule.

I stopped being lazy and took the time to set up an AWS ec2 instance as a proxy for FFXIV yesterday. I cannot believe I have to do this again to fix a connection issue with a game, but whatever.

Quote:

---

Originally Posted by aerimisia

and AT&T blocks all of the work arounds that would potentially allow you to use one on your PS.

---

Honestly confused why or what they block. From my research they only block ports that spammers abuse.

Quote:

---

Originally Posted by Krojack

Honestly confused why or what they block. From my research they only block ports that spammers abuse.

---

I’m not really sure myself. I know they don’t like you tampering with their equipment and they have something in their routers that blocks you from putting a VPN on the router itself. And when I tried creating a bridge to use my laptop for a LAN connection to use the VPN on my laptop I couldnt do it but my friend who has a different ISP was able to do it.

Quote:

---

Originally Posted by aerimisia

I’m not really sure myself. I know they don’t like you tampering with their equipment and they have something in their routers that blocks you from putting a VPN on the router itself. And when I tried creating a bridge to use my laptop for a LAN connection to use the VPN on my laptop I couldnt do it but my friend who has a different ISP was able to do it.

---

Weird. The password to mine is right on the back and I can do whatever I want in it. I run all sorts of in-home servers and even a Plex server too.

Quote:

---

Originally Posted by Illyrian94

As expected, AT&T's response is to point a finger back at Square and NTT and say we need to reach out to them.

https://forums.att.com/conversations...05de56a01bcff0

I'm going to open yet another support ticket with SE. Not that I expect anything to come of it, but I figure I can at least be a continual support issue for them if they're going to continue to refuse to acknowledge the issue and bounce us back and forth to AT&T. What a joke of a situation.

---

Unfortunately as others pointed out it seems all we can do is speak with our wallets. My sub was due to renew next week and I cancelled. Placing another ticket just got SE to tell me to check my ethernet cable despite referencing this post.

Quote:

---

Originally Posted by Krojack

Weird. The password to mine is right on the back and I can do whatever I want in it. I run all sorts of in-home servers and even a Plex server too.

---

Could it be we have different models? I’ve had fiber pretty much since they started rolling it out so maybe my router is older? I don’t know a whole lot about this stuff to be honest and I could have definitely messed up somewhere, but my friend walked me through it and he works in IT so I trust he knows what he’s doing

Well, we're 10 days in now with absolutely no relief. Either the parties involved dont care or they are not competent enough to deal with the problem. RIP
I'm just so happy that we have new content that we can't play!

Honestly, I *never* expected them to fix it before 6.4 dropped. Understandably, their focus was on getting the patch out. Now that the patch is out, no more excuses. This should be next on the "get fixed" list. There's so many people who *can't play your new content* Square....

Two weeks without lag relief. Posting to support this thread. 8 days till my sub is donzo

Quote:

---

Originally Posted by aerimisia

Could it be we have different models? I’ve had fiber pretty much since they started rolling it out so maybe my router is older? I don’t know a whole lot about this stuff to be honest and I could have definitely messed up somewhere, but my friend walked me through it and he works in IT so I trust he knows what he’s doing

---

I have the following one supplied by them. I can even enable IP Passthrough which means the modem just acts as the connection. All data can be passed though to your own personal router if your choice. The modem at that point is pretty much invisible to anything on your network. It's no longer your router, firewall, wifi or anything. Your new router does all that.

https://www.amazon.com/802-11a-Wirel.../dp/B08PCRYJPW

Been checking back in with this thread for any news and updates on the lag situation. Nord VPN has been working really well since Cloudflare took a shit and stopped working. Sucks that I had to do this and spend money to play something I am already subbing to but it's like Sweet Tooth said in Twisted Metal Black, "Let's face it boys and girls. A man has to have his priorities." Quality time with my dearest is essential so I gotta do what I gotta do.

replying to add more info. not on ATT, on centurylink fiber in midwest US, getting routed through bad NTT nodes as well. getting tons of packet loss to game, just started yesterday

Unfortunate that the issue seems to be spreading and getting worse. All with no proper acknowledgement from Square, either. They responded to my support ticket yesterday by saying they were "aware of the issue" and directing me to submit another tracert result to the form the community manager posted earlier--so basically the same response we've been getting this whole time.

I'll post my daily "still broke" screenshot later today like usual. Cloudflare Warp is thankfully still working for me, but once my currently canceled sub runs out I'll probably be done with the game for awhile unless Square decides to finally bless us with a proper response.

I hope they do get on the ball Illy; as for me, got Aetherfont done this morning and my island's set up for Momocon, so I'll be back after the con. It's aggravating that me and 'Neko can't play together until Square/NTT gets their act together. He works, I don't have a job at the moment (tho working on that) so I can get some stuff done early in the day if I get up in time.

(And off topic; if anyone's going to Momocon, look out for this miqot'e in human form! I'm planning on showing up at the FFXIV panel and the photoshoot.)

When does the lag usually get better in the evenings? Last night I go a bit of a reprieve around 11:30 PM EST. Thought that maybe they did something, but heard that there was widespread DC'ing early this morning.

My Traceroute showed that the issue was in NTT specifically. First hop on their network had good ping, but after the first, it dies. Probably why AT&T isn't doing anything about it. Ironically, this happened in the past I found on reddit. 2 years ago same issue.

Kind of feel this ATT NTT node issue along with other ISPs will last till next month because no way they will fix this problem before Savage next week. kinda sucks that the ff14 Twitter page is just pointing fingers at our isp to say something yet most of the responses have been going to square or we can't really contact NTT to fix the problem but I'm just going to stick with the lag since I can't afford a VPN. sometimes cloudflare crashes around midnight for me. so rather just suffer with the lag till it fixes itself. I personally feel they throttled us hard and knew half of us were hyped for 6.4 and I personally feel they sabotage the severs with lag.

Code:

---

  3    6 ms    3 ms    3 ms  71.155.29.140
  4    *        *        *    Request timed out.
  5    13 ms    7 ms    7 ms  32.130.25.65
  6    8 ms    6 ms    7 ms  ae-5.a00.dllstx14.us.bb.gin.ntt.net [129.250.8.237]
  7    9 ms    6 ms    8 ms  ae-2.r20.dllstx14.us.bb.gin.ntt.net [129.250.4.9]
  8    *        *        *    Request timed out.
  9    48 ms    43 ms    45 ms  ae-5.a01.scrmca03.us.bb.gin.ntt.net [129.250.2.7]
 10    62 ms    66 ms    57 ms  ae-1.a00.scrmca03.us.bb.gin.ntt.net [129.250.4.76]
 11    45 ms    45 ms    51 ms  xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
 12    75 ms    68 ms    65 ms  204.2.29.241
 13    45 ms    44 ms    44 ms  204.2.29.6

---

The latency is - as others have alluded to - starting later in the day now. I'm not holding my breath though. It's still not what I would get on a normal play session prior to 05/15. Let's see how it holds later.

While the AT&T response with the most info is not from an employee, but from an "expert", I do tend to agree with what they and AT&T reps say: it's on NTT. I really do hope it's not some sort of dumb money spat between the two, that would just make me so sad to suffer because of greed.

Issue started popping up today in the early AM around 5-7am EST for me instead of just during "prime" time.