Bad lag again with NTT's NA server network

[ntt_routing_is_an_issue]

Well I tried to follow up with NTT to press a bit more but this person is at least making it the ISP service's issue.

---------------------------------------------------------------

Hey Victor,

Can you check this out for me if you have time?

In your email:

Your ping test you have a Response time of:

PING 204.2.29.87 (204.2.29.87): 56 data bytes
64 bytes from 204.2.29.87: icmp_seq=0 ttl=57 time=40.044 ms

In my tracert I have a response time of:
11 116 ms 117 ms * ae-5.a01.scrmca03.us.bb.gin.ntt.net [129.250.2.7] <<< The ms increase between Hop 8 and 11 is expected as the traffic traverses the NTT network between Dallas, TX to Sacramento, CA


Isn’t that a little strange that we are both from Texas, and getting vastly different ms response?

Also, I ran two different Tracerts to a hop earlier in the same route and get this:

Tracing route to ae-1.a00.scrmca03.us.bb.gin.ntt.net [129.250.4.76]
over a maximum of 30 hops:


4 * * * Request timed out.
5 * * * Request timed out.
6 * * * Request timed out.
7 29 ms 30 ms 30 ms 32.130.25.67
8 26 ms 30 ms 26 ms ae-5.a00.dllstx14.us.bb.gin.ntt.net [129.250.8.237]
9 * 26 ms 26 ms ae-2.r20.dllstx14.us.bb.gin.ntt.net [129.250.4.9]
10 112 ms 115 ms * ae-2.r24.lsanca07.us.bb.gin.ntt.net [129.250.7.69]
11 141 ms 123 ms 142 ms ae-5.a01.scrmca03.us.bb.gin.ntt.net [129.250.2.7]
12 72 ms 71 ms 73 ms ae-1.a00.scrmca03.us.bb.gin.ntt.net [129.250.4.76]

In the full route, I get this:
Tracing route to 204.2.29.87 over a maximum of 30 hops


4 * * * Request timed out.
5 * * * Request timed out.
6 * * * Request timed out.
7 30 ms 31 ms 31 ms 32.130.25.67
8 31 ms 27 ms 31 ms ae-5.a00.dllstx14.us.bb.gin.ntt.net [129.250.8.237]
9 * * * Request timed out.
10 111 ms * * ae-2.r24.lsanca07.us.bb.gin.ntt.net [129.250.7.69]
11 118 ms 117 ms 118 ms ae-5.a01.scrmca03.us.bb.gin.ntt.net [129.250.2.7]
12 * 175 ms 117 ms ae-1.a00.scrmca03.us.bb.gin.ntt.net [129.250.4.76]
13 119 ms 118 ms 119 ms xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18]
14 * * * Request timed out.
15 * 117 ms 117 ms 204.2.29.87


So that doesn’t match the description of the original email where:
8 40 ms 36 ms 30 ms ae-5.a00.dllstx14.us.bb.gin.ntt.net [129.250.8.237] <<< This is the ATT node where the traffic is picked up by NTT
9 * * * Request timed out. <<< This is a node that is not responding to ICMP, does not indicate packet loss or latency
10 * * * Request timed out. <<< This is a node that is not responding to ICMP, does not indicate packet loss or latency
11 116 ms 117 ms * ae-5.a01.scrmca03.us.bb.gin.ntt.net [129.250.2.7] <<< The ms increase between Hop 8 and 11 is expected as the traffic traverses the NTT network between Dallas, TX to Sacramento, CA
12 118 ms 117 ms 116 ms ae-1.a00.scrmca03.us.bb.gin.ntt.net [129.250.4.76] <<< The ms stayed consistent
13 117 ms 119 ms * xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18] <<< The ms stayed constsent


It seems like Hops 9 and 10 are:
9 * 26 ms 26 ms ae-2.r20.dllstx14.us.bb.gin.ntt.net [129.250.4.9]
10 112 ms 115 ms * ae-2.r24.lsanca07.us.bb.gin.ntt.net [129.250.7.69]

And yet we can see it cannot always display them reliably. Can you see if there is something strange going on there? Or is this normal behavior?

-----------------------------------------------

Good day,

Per my previous email, I do suggest contacting your upstream provider to determine the reliability issue you are working on.
See inline for response, in red, below:

---
Victor Nadalalicea
Network Analyst, Global IP NOC


Isn’t that a little strange that we are both from Texas, and getting vastly different ms response?

>>> Not strange at all, but more expected. Pings and traceroutes are different in that sense yes. Traceroute calculates how many hops it takes to its destination, and how long each hop takes. With Ping, this app just calculates the time per echo reply responses. Therefore, both numbers will always result in different ms readings. You should never receive the same results in ms.


So that doesn’t match the description of the original email where:
8 40 ms 36 ms 30 ms ae-5.a00.dllstx14.us.bb.gin.ntt.net [129.250.8.237] <<< This is the ATT node where the traffic is picked up by NTT
9 * * * Request timed out. <<< This is a node that is not responding to ICMP, does not indicate packet loss or latency
10 * * * Request timed out. <<< This is a node that is not responding to ICMP, does not indicate packet loss or latency
11 116 ms 117 ms * ae-5.a01.scrmca03.us.bb.gin.ntt.net [129.250.2.7] <<< The ms increase between Hop 8 and 11 is expected as the traffic traverses the NTT network between Dallas, TX to Sacramento, CA
12 118 ms 117 ms 116 ms ae-1.a00.scrmca03.us.bb.gin.ntt.net [129.250.4.76] <<< The ms stayed consistent
13 117 ms 119 ms * xe-0-0-5-0.a00.scrmca03.us.ce.gin.ntt.net [128.241.2.18] <<< The ms stayed constsent

>>> Traceroute to different destination IP addresses, you should expect different paths, because the destination network address is different and depending how the route path algorithms are calculated will determine the path it takes. The above traceroutes to two different destination addresses are expected.

And yet we can see it cannot always display them reliably. Can you see if there is something strange going on there? Or is this normal behavior?

>>> We only have visibility within our network, and I have shown there is no issue that would cause your reliability issues.

-----------------------------------------------------------

I don't think they're losing sleep over this.

This is largely what I expect. When I was experiencing this on both AT&T (5 gig fiber) and Spectrum (1 gig cable) SE told me to call the ISP. I knew what the response from AT&T and Spectrum would be, and I agree with them since as I leave both their networks I have good latency and no packet loss. AT&T eventually escalated me to some engineers in Dallas who confirmed what I already knew, the issue is inside NTT. This is evidenced by the fact that I have no problems with the first NTT hop in Dallas. The second hop, also in Dallas, can from time to time introduce some latency, but the packet loss occurs when transiting to NTT in California. By this time, we are already inside NTT's network and it completely outside of our ISP's control. NTT announces their routes, and the ISP uses what is broadcast. It's just a stall tactic that has unfortunately become common in the industry to save face. Also interesting that I have been complaining about this since early April and it wasn't until some site wrote an article about it that SE finally calls NTT to work the issue.

[Ath192]

This is largely what I expect. When I was experiencing this on both AT&T (5 gig fiber) and Spectrum (1 gig cable) SE told me to call the ISP. I knew what the response from AT&T and Spectrum would be, and I agree with them since as I leave both their networks I have good latency and no packet loss. AT&T eventually escalated me to some engineers in Dallas who confirmed what I already knew, the issue is inside NTT. This is evidenced by the fact that I have no problems with the first NTT hop in Dallas. The second hop, also in Dallas, can from time to time introduce some latency, but the packet loss occurs when transiting to NTT in California. By this time, we are already inside NTT's network and it completely outside of our ISP's control. NTT announces their routes, and the ISP uses what is broadcast. It's just a stall tactic that has unfortunately become common in the industry to save face. Also interesting that I have been complaining about this since early April and it wasn't until some site wrote an article about it that SE finally calls NTT to work the issue.

IDK that SE will get anywhere. Lets hope they do though. I don't see the difference between them telling an intern to see what's up and us doing the same.

PIA has decent prices I guess....

SE peeps! just move the Datacenter to Dallas plz. xD

[ntt_routing_is_an_issue]

IDK that SE will get anywhere. Lets hope they do though. I don't see the difference between them telling an intern to see what's up and us doing the same.

PIA has decent prices I guess....

SE peeps! just move the Datacenter to Dallas plz. xD

SE has more pull since they are the customer of NTT. If random Joe calls NTT they are going to brush it off, but if SE calls it's an issue because there are SLA's in place.

[Ath192]

SE has more pull since they are the customer of NTT. If random Joe calls NTT they are going to brush it off, but if SE calls it's an issue because there are SLA's in place.

Honestly maybe another move for SE could be to contact the Datacenter that is hosting their equipment to see if they can reverse ping to several locations East of them and find the issue and work on it from their end. I bet SE pays them and doesn't own the Datacenter outright but I could be wrong, so it would be in their interest to keep a big customer.

[ntt_routing_is_an_issue]

Honestly maybe another move for SE could be to contact the Datacenter that is hosting their equipment to see if they can reverse ping to several locations East of them and find the issue and work on it from their end. I bet SE pays them and doesn't own the Datacenter outright but I could be wrong, so it would be in their interest to keep a big customer.

It's leased. WoW uses the same datacenter from the looks of things. Unfortunately, I think most of the time the only thing that motivates companies to dig into these issues is when it affects the revenue stream, or gets brought to light by a publication that paints them with a negative stroke. Having an article written about you forces your hand to respond. I'd guess that most people in these threads that are threatening to cancel their subs actually won't, and will just ride it out for the one day its playable which resets the outrage timer until the next issue.

Your proposal was honestly kind of what I was expecting the response to be, but the ONE support person I even got to mention tracerouting back to me cited an IP that wasn't anywhere in my route map, so its likely they just did it from their workstation and not from within the segment the game servers live.

[Ath192]

It's leased. WoW uses the same datacenter from the looks of things. Unfortunately, I think most of the time the only thing that motivates companies to dig into these issues is when it affects the revenue stream, or gets brought to light by a publication that paints them with a negative stroke. Having an article written about you forces your hand to respond. I'd guess that most people in these threads that are threatening to cancel their subs actually won't, and will just ride it out for the one day its playable which resets the outrage timer until the next issue.

Your proposal was honestly kind of what I was expecting the response to be, but the ONE support person I even got to mention tracerouting back to me cited an IP that wasn't anywhere in my route map, so its likely they just did it from their workstation and not from within the segment the game servers live.

Are WoW players having the same issue?

Doesn't look like it. They seem to have different IPs/Location

https://us.battle.net/support/en/article/7870

[ntt_routing_is_an_issue]

Are WoW players having the same issue?

Doesn't look like it. They seem to have different IPs/Location

https://us.battle.net/support/en/article/7870

Yep, looks like when they had servers hosted in Dallas they used NTT, but maybe 2 years ago that changed when they move to Chicago and AT&T. Updating my brain database.

[Janyd]

SE has more pull since they are the customer of NTT. If random Joe calls NTT they are going to brush it off, but if SE calls it's an issue because there are SLA's in place.

Thankfully, if yesterday's social media posts (and the posts in this thread from the Community Reps) from FFXIV are any indicator, they are likely doing just that as we speak.

[ntt_routing_is_an_issue]

Thankfully, if yesterday's social media posts (and the posts in this thread from the Community Reps) from FFXIV are any indicator, they are likely doing just that as we speak.

One can hope. Guts me when I can't raid because the game is unplayable. More so that I let 7 other people down due to something out of my control.