Sudden Lag & Packet Loss

[Shibi]

I think some of us here are experiencing different issues.

No doubt. And as always, MTR's like this can potentially help see the problems for what they are.

I also have frontier, but am only experiencing this issue with FFXIV. There's a bunch of packet loss when the connection moves over to the NTT servers on the route to SE servers.

ICMP is normally QOSed down in priority to real data, so the loss numbers are not too surprising, although Level3 is implicated there as the cause - as usual.

The STD Deviation on your packets is the more useful indication in this case, and the worst you show there is 4ms - which is nothing. Sadly it looks like it's at 10pm, so maybe not as informative as something done at say 7pm when the Std Deviation will be much more informative.

Also connecting to my VPN server does not resolve the issue, as it's still routed through the affected NTT servers in cali.

Sadly NTT provide DC services to SE, so it's going to be impossible to skip those. And getting SE involved with their provider (NTT) needs hard evidence - and a lot of it - in the form of traceroutes or better MTRs while the lag is happening as they asked for.



In general advice, to everyone, as always the "me too" posts are pretty meaningless and won't solve the problem. It's the best advice anyone can give to start up a traceroute or better yet an MTR during peak and get some hard evidence, post that and slowly (because it does take time to gather enough evidence) everyone can hone in on where the problem is.

[Joker119]

Unfortunately there isn't really a temporary fix for the issue at hand.

The issue seems to be the NTT server network that connects to the datacenter can't handle the traffic load players are putting on it (this likely only became a recent problem because many more people are playing with the 6.2 patch release than were playing before).
It won't affect everyone, as my friend in Asia can play on NA servers without any issue at all, because her route to the data center doesn't take the NTT network.

In simple terms

Think of the datacenter as an intersection with several roads (ISP networks) leading into it.
The intersection itself isn't to blame if one of the roads is only 2 lanes wide and backs up, when all the other roads are 4 lanes wide and have no issues.
Your route to the intersection (data center) will determine your connection issues (or lack thereof). Unfortunately it's not the responsibility (or even realistically possible) of your individual ISP to stop using the broken route. Nor is it possible for you yourself to change your route.
The people who maintain the road (NTT) need to widen it (increase network capacity) in order to resolve the issue, or take the affected server (since it seems to be caused by one specific NTT server along the route, currently) off the available route (close the road so people take the other, bigger ones that have less traffic).

However from what i've been able to gather, NTT doesn't seem to give half of a shit about the issue whatsoever, and are happy to just sit on their thumbs doing nothing.
Since the data center being served is SE's, they would need to be the ones to push NTT to a resolution, or work with their data center's ISP to take NTT off the routing list.

That being said, you *may* be able to work around (literally) the issue, by using a VPS located somewhere that isn't close to you - such as Northern Cali, WA, OR or elsewhere.

My VPN in NV has the same issue, however neither of my servers in MO do, so I'm probably going to setup a vpn server for myself on one of those to use.

[Joker119]

I was actively experiencing the issue when I took that MTR.
The problem isn't that packets are spiking in latency, it's that they are dropped entirely. AFAIK MTR doesn't track that in StdDev.

It shows a similar result simply using ``ping`` aswell, after 100 pings, the avg latency is 38-40, StdDev of around 3-6, but 15-20% packet loss.

Also for what it's worth - a StdDev of 4ms is quadrouple what it is right now, at 0230 with the problem nowhere to be found.

I've been in SE's place, where the datacenter has a bad network server servicing my dedi and it took me 9 months to resolve the issue. In the end, it turned out to be a bad NIC in one of their servers that was causing the issue, even though they spent the previous 9 months blaming me and my dedi, and then blaming people's ISPs, and then blaming a backbone provider, and then blaming firewall rules, etc, etc. The whole time the MTR showed something very similar to what's going on here.

It seems SE was on the verge of, if not already, getting ontop of NTT about it back in 2018, then I'm assuming player count fell ShB was ending, then it started back up when EW released, died back down as people cleared the MSQ and content and went on to other games more, etc. And has once again come up because of the new patch drawing in more people.

It's likely that SE was bugging NTT about the issue, NTT was stalling because that's what all ISP/DC providers do because they don't actually want to pay network engineers to fix anything, and the problem 'went away on it's own' because there was less traffic, so SE lost the incentive to keep ontop of NTT about the issue and so they just never actually addressed it.

[Brygd]

I'm already being impacted at 3:30PM PST today. I really hope this gets fixed soon. The game is unplayable at peak hours (aka when I get home from work, before I get to sleep).

[VulpessRahl]

Setting up a VPN seems to have patched the problem for me so far.

[SerenityAbyss]

Got VPN right now as a workaround. So far, it works, but waiting to try it on the weekdays at peak hours to see if it holds up.

[Brygd]

Setting up a VPN seems to have patched the problem for me so far.

Do you have a VPN service you recommend?

[sr7olsniper]

Do you have a VPN service you recommend?

Malwarebynes, expressVPN or I use the one that comes with Norton.

[mishakeet]

There is a post on the Frontier Reddit discussing the situation: https://www.reddit.com/r/frontierfio..._high_latency/

[Nevrand]

Throwing my hat in the ring just to add one more voice. Much like everyone else here I am on the Frontier network and I live in SoCal. Packet Loss started for me on Wednesday afternoon/evening, just about 7 PM. Since then every afternoon/evening I have dealt with packet loss. Pinging the NA datacenter as well as Cloudflare's DNS sever (1.1.1.1) shows the packet loss. Interestingly enough, pinging Google's DNS (8.8.8.8) yields no such packet loss. I have run several sets of 3 600 packet ping tests and the result has almost always been the same. 0% packet loss to Google's DNS, and varying amounts of packet loss to Cloudflare's DNS server as well as FFXIV's NA server. Below is the result of one such test.

Ping statistics for 204.2.229.116:
    Packets: Sent = 600, Received = 501, Lost = 99 (16% loss),
Approximate round trip times in milli-seconds:
    Minimum = 32ms, Maximum = 48ms, Average = 33ms

Ping statistics for 1.1.1.1:
    Packets: Sent = 600, Received = 517, Lost = 83 (13% loss),
Approximate round trip times in milli-seconds:
    Minimum = 22ms, Maximum = 38ms, Average = 24ms

Ping statistics for 8.8.8.8:
    Packets: Sent = 600, Received = 600, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 5ms, Maximum = 22ms, Average = 7ms

Another interesting thing about this issue is that I notice the packet loss when playing Destiny 2, but didn't notice any such loss when playing League of Legends. Also whether my Discord is affected has varied. I'd love to run ping tests to these servers, but I couldn't find specific IP's tied to Destiny 2 or Discord. League of Legends' NA server IP is public, but does not reply to ping tests from the attempts I've made.

My assumption with Discord is it depends where the server is located for the voice call. This would also line up with seeing packet loss in Destiny 2 but not League of Legends because Destiny 2 has servers on the West Coast while League of Legends NA servers are in Chicago. This could be mere coincidence though, but the results have been consistent over several days.

Hopefully this issue is fixed sooner rather than later. Using a VPN has worked for me, but is still not ideal due to ping fluctuations. Also it'd be nice if things just worked like they're supposed to.