Latency issues

[Spiroth]

Last 3 weeks have been a disaster.
Today we had to call our raid off because it was literally impossible to play.

After alot of time not getting into Coil, I decide to try it seriously 2 weeks ago, because I want to experience the story, and had time cause broken leg.
I found a static to go on from Turn 6 that I had stopped since a year ago, but can't play. 1st I was a liability, then I told them I couldn't come cause lag, and they replaced me of course.....

[Raist]

They don't care.Notice how there isn't 1 single post of anyone managing to fix their 90000/90002 errors on their end on these forums.All you will see is players suffering from these errors then....silence.....

If this problem was our ISP's like the white knights want us to believe,then you'd think we'd have quite a few posts from players on this board saying they fixed it on their end.

Guessing you haven't been tracking this problem for very long. There have been numerous posts on these forums and abroad for over a year where people have resolved latency/connectivity issues after addressing routing problems by either working with their ISP or using a VPN. SE had nothing to do with fixing their problems...it was purely alleviated by adjusting parameters of the route between the player and Ormuco (SE's ISP). A quick google hit threads on this topic going back to December 2013. There are numerous posts discussing progress with these very processes for various services that have been mentioned time and again. The same principles that applied over a decade ago still apply to these more recent posts on the matter. It hasn't changed because our infrastructure in North America is still in really bad shape. Here is a link to one such thread on these forums:

http://forum.square-enix.com/ffxiv/t...-user-with-lag

It was linked in a thread at another site here:

http://www.wutaicompany.com/reddit/1...20lodestone%5D


And it isn't White-Knighting. It's based on precedent. We've seen this play out time and time again. It is a very common and well known source of trouble for web based services in general and not just this game service. There have been public filings with regulatory authorities multiple times over the years in attempts to put pressure on the ISP's to clean things up. Articles on the topic have been posted here multiple times, even have pasted the slides and such in posts here attempting to demonstrate it. For starters, here are links to a series of blog posts from Level3 on the matter:

http://blog.level3.com/category/open-internet/page/2/
http://blog.level3.com/category/open-internet/


If people would take the time to read them, it will explain a lot of it in detail...some complete with graphics like this from the "Not Neutrality" post:

Each number shows utilization at one of the interconnection locations in various cities throughout the United States between Level 3 and the LECs.
Utilization above 85% indicates the LEC is causing congestion in that city by refusing to add interconnection capacity

People have been dealing with it off and on for at least as long as I've been gaming online, and that goes back to 1996. Back then, broadband was just getting to be a mainstream thing for household use, and we were dealing with premium bandwidth packages below 2Mb/s downstream and no more than 512Mb/s upstream--nothing compared to the bandwidth offered today. Unfortunately, you can see some markets still using some of that same ~20 year old equipment that can't even properly support the bandwidth they're now selling to us. I tracked down one such device at our head-end last spring--it was running a firmware released when Bill Clinton was President, and only bonding 1/2 the channels our DOCSIS 3 connections needed for the speeds they were selling--it has since been replaced. The problem back than is the same as it is now...getting people to hold their ISP's accountable. The same thing that was being discussed in the thread linked above.

Sitting around waiting for SE to fix the widescale infrastructure problems is not going to work...they don't have the leverage, as they aren't the ones paying our bills for our internet access. There is a simple way to verify if it may indeed be a routing issue(apply the principles to this game's addresses), and options for either skirting around the problem (VPN) or working towards actually resolving those issues (Tier3 escalation), if you are willing to put forth the effort. Many have done so in the past and benefited. The problem is far too many people just come here to vent and never take the initiative to actually do something about it.

[Kaze3434]

all very true info

sadly raist, i gotta say it; give up. this entire thread is dead set on refusing to listen to anything besides saying "Its not my fault, and i already checked with my isp, so its SE!" and that is the end of story for them. making informed comments is beyond them. its kinda sad that you actually had to go as far as you have to try to explain how the internet works, and people still seem to think its just magic invisible tubes that always work forever.

[Rofel]

I'm having the same problem in the last two, three weaks. I can play only a couple of hours / day and some nights (mostly of them) I can't even do the daily duty roulette. Playing on PS3, so WTFast is not an option.

Server: Sargatanas.

[Khemorex]

i have these weird lags a lot lately specially in expert roulettes , no lag in high lvl roulette tho :/ also i lag in gold saucer but not in uldah... go figure...


edit : i did a tracert , and i got 110-150 miliseconds , that dosnt help me if it lags in expert roulette and not in high lvl roulette .. :P

[Madorin]

(Hopefully this is the right place to post; ) I don't know a lot about computers, so if I've said something stupid I'm sorry :'D;

I've also been having lag problems with FFXIV that never happened to me before I got the new patch, like the game can become a bit choppy or freeze for a bit (This happens a lot in FATEs with lots of players and when it restarts I am nearly dead X'D; ) In Mor Dhona the game got to 0fps and completely crashed and closed (I forgot to note the error message it gave me though OTL) ; My frame rate before the patch was 120fps, but now it's only at 20-30fps ; v ;''

I used my NVIDIA Geforce to make my settings 'optimal' (it ran at 120fps originally with all the settings at highest) alongside downloading WTFast, got something called Leatrix Latency Fix and then disabled powersaving on my network cards as I read something saying doing these things can reduce lag, but it hasn't changed anything;; (except the graphics thing, now the game is less pretty XD ) I checked my broadband speed and its ping is 94m/s, so I'm pretty much assuming the lag is from the game? TT 7 TT;;

I have an Asus G750JX with hardly any of the storage taken up, so uh;; I don't think it's being slowed down by anything on my laptop, either;; So I can't really think of anything else right now except the patch;;

I'm playing in London, England, on the EU server Moogle.

[Bariaus]

Kinda laughable, but this is the thread that SE support linked me as a way to say: 'We cannot help you further. Check this link.' And yes I'm being literal.

So yeah, I'm new (in the forums), and I'm here just because of this lag thing problem. Nice to meet you everyone.

Info:

Country: Chile
ISP: VTR Globalcom S.A
Traceroute explained: First 5 local servers avg. 20ms. Rest of the international servers (Level3 and FF XIV IPs) avg. 250ms. All of them located in the US and Canada.
My rig: Core i7 3770 3.4ghz, NVIDIA Geforce GTX 570 2gb, 8gb RAM, 1TB HDD, Windows 7 premium. Not getting into details because this will suffice for those talking about 'graphics lag'.
Network hardware context: ISP Cable modem Motorola Surfboard SB5101 connected to router TP-LINK TL-WR740N. Router has a network cable connected to my rig.

Situation: I've been playing this game for about 1 year. 3 weeks or so ago and never had any issues about lag. We moved to another house (the one next to where I lived before lol). We kept the same service, but obviously they re-wired everything in this new house, so I'm guessing they changed the IP of my modem which I kept too (my ISP has static IP which changes every 2 months or so). This is where the problems started, specifically, the 'everyone's problem' 5 or so seconds of lag spike, which is constant. At least for me, this situation happens from 12pm day time (gmt -3) to 12pm night time (gmt -3). Peak goes from 5pm day time to 10pm night time. This is when games is practically unplayable and yes I'm not being a whiner. It tecnically is unplayable.

Since I play lots of instances, specially coils, this situation is particular, because like everyone else states, the lag is notoriously higher and constant in these situations, and at least for me, it does not discriminate time. It can happen in AM or PM time, even if when I'm outside instances, the lag is present at the times previously stated. The odds are fewer in AM time, but they happen too.

So, started to wonder and investigate about all this problem, and did a tracert, so I could check the path my IP is taking to reach the game and also check the ping: Extremely low pings in first 5 IPs which I scanned, and they were locals. Then it jumps to 130 in a CDN located in US. After this we reach our beloved friend Level3, where my ping reaches 260-300ms. I've read about this and short story, for mmorpg, it is bearable, at least to not 'feel' a 5 second lag spike, and constant. Investigated further, and realized that Level3 is congested, meaning that lots of IPs are passing through this CDN, causing - something we call here - like a 'bottleneck'.

Finally, I began to gather all the pieces:

- ISPs (at least in my puny little country) doest not have control nor contact over external CDNs / ISPs to command them to change the route of an specific IP. They do have control over local servers.
- Level3 is congested
- Unbearable lag specially in afternoon - night.
- .... what the hell is SE doing about this? Can they contact these US / Canadian ISPs and CDNs about this issue?

Idea:

Since US ISPs can also have control over where are their IPs routing, they choose most of em to pass through Level3 (because it's local you know, duh), and since in the afternoon people leave work, they are tired and watch lots of Netflix and such, in this case Level3 collapses. ISPs point that some CDNs do not have the infrastructure, and CDNs points that ISPs are organized to pass most of their IPs through their servers. Yay. Ah yes, and for me since I have similar hour time as US, I get lag because it fits in the same time US ISPs clients are using internet the most. Oh..uh, where is SE in all of this?



What I've tried (1 by 1, simultaneously, rebooting, etc.):

- Port Forwarding.
- Google DNS.
- 3rd party programs to check best DNSs in the area.
- Leatrix latency fix.
- 3rd party VPNs like Battleping and WTFast.
- Played directly with modem, avoiding router.

None of the above worked, and believe when I say I tried like 10 times each of these.

==

So. Read some stuff in this forum, also in external forums and it seems to be some people that knows a lot about all of this. Hopefully they can answer some of my questions in this post, above all, regarding the role of SE and their servers in all of this.


P.S - A little warning about my English doesn't hurt.

[Raist]

@ Bariaus,

Is that VTR the Banda Ancha group or the main GlobalCom S.A. ? Having trouble nailing it down exactly... VTR Globalcom seems to come up with multiple brands, most of which have outdated references for routing numbers. Banda Ancha seems to be the only current one registered, and it is covering a huge list of addresses for Chile. ASN's are linked to multiple banks of IP's starting with 190, 200 , or 201... each one has dozens of blocks under them (200.74.xxx.xxx, 200.73.xxx.xxx, 200.30.xxx.xxx, etc.). Could narrow it down more if I had an actual IP address. If we go with this ASN though, they do have another choice besides Level3 if they can't work something out with them to clean up the routes.

They should be able to work something out with Level3 though. That particular ASN (AS22047) shows that they peer almost 50/50 between Level3 and Telefonica. They are likely defaulting to Level3 over Telefonica because their analytics are estimating Level3 would be the better choice because it may get them more directly into global transit--Telefonica may wind up switching off to another peer for more global access in some cases, depending on their policies...which may be part of why they are opting to use Level3.

They should be able to switch you off to Telefonica though...which may prove a better choice in the long run, depending on how they wind up routing you to get to Ormuco in Montreal. Telefonica peers with multiple providers that Ormuco peers directly with: TATA, Cogent, TiNet/SPA, as well as Level3 (the only one not covered is Alter.Net/Verizon Business). Telefonica has multiple landing points in/around Chile, so they should be able to get you on to their network to see if things improve:

https://www.globalsolutions.telefoni...ional-network/ (this link sometimes doesn't load initially, may have to refresh your browser)


Direct Link to Full Screen map: https://www.globalsolutions.telefoni.../27248/map.jpg

That may seem like going around the elbow to get to the thumb, but it may result in more stable latency if it is able to get you around the congested hops within Level3's network. The catch will likely be getting through to someone further up the administrative chain to look into the options though.

[Bariaus]

@Raist, thanks for taking your time in explaining this. Btw where do you get all this info, haha!

So much information, and still so many thing to say... where to start:

VTR and Telefonica are the 2 big ISP companies here. We've always been with VTR, and it's just my guess, but could it be that my IP ( auto-censoring 201.xxx.xx.xxx), that belongs to that ASN, is routing through Level3 because VTR is avoiding Telefonica since they are competing companies? I Think I understood that VTR knows this ASN and they are full aware that my IP is travelling through Level3 then. If neither technical support, nor the 'gamer' area can resolve the issue (no upper tier after this, they told me), then I guess I'm screwed.

I don't know what else to say really, but it would be helpful if you explained a little bit more this ASN topic, and how my ISP is fully aware of which path are their IPs routing. Will investigate further on the net too.

I'm trying to get the idea that no one here is the 'culprit'. My ISP in this case, chose Level3 because it's more direct, maybe they're not aware that this CDN is that congested, because they don't care about the reality outside their local servers. CDNs are not the culprit either because.. they can't decide which IP goes through their servers, or can they? Not even SE is the culprit since they are just another server and last destination, and the congestion is in Level3, not in Ormuco or whatever SE is working with. And lastly but no less, can SE do his thing with Level3, assuming that the latter can actually decide which IPs routes through their servers?

Still confused, but saw some light with the ASN - ISP info.


Thanks again!

Edit: You said that this ASN points to Level3 or Telefonica. If, in fact, VTR can change this, do they have 'power' to decide which of all this list you mentioned (cogent, TATA, etc.) is my IP directed? I'm saying ths because it can still route through Level3. It is 'pure luck' or how does it work? Who controls this? Sorry about all the questions, but I really do not know. And if I knew and it is on my ISP, then it's time to call again then. Thanks again @Raist.

[Raist]

In some cases, the last mile ISP's (your ISP in this case) may not be aware of the congestion further upstream. They should be aware if it is happening at the more localized exchange though. It happens often simply because the peering agreement is not set up with enough bandwidth to handle all the traffic they are trying to send through the exchange. Peering is often basically just an agreement that the companies using the exchange will share the costs of maintaining the exchange and such so they can all use it. There are a lot of "public" exchanges that have so many people coming through them that they simply get overloaded, and then there are the more "private" ones where money may actually exchange hands for access (which ideally would cut the congestion, but even then sometimes they don't purchase enough bandwidth). So, in many cases there actually is something an ISP can do to reduce the congestion--but it may mean spending money to do so, so it doesn't happen.

Your ISP peers with both companies. Which one you are assigned is determined by the policies in play, both for the peering agreement and for how routing is analyzed/implemented. They can manually create routing entries or they can use routing protocols that try to calculate/predict which routes may put forth the best effort (where the Autonomous System Numbers typically come into play--to share route details). Either system can run into problems, and simply get you stuck on a bad path. Things happen... bad weather, sudden shifts in usage patterns--sometimes the network just can't be adjusted fast enough and the pipes get clogged. I've looked up some of the calculated metrics between me and Montreal in the past and seen some routes report anywhere from 130-800+, but know some of them almost never run high enough to be pushed that far up the priority list when I've been on them. The values probably got assigned in a period when traffic was heavy, but then things evened out and those tables are no longer accurately reflecting the states of those routes. When it works right, it's amazing... but when it gets it wrong, it can be pretty bad.

Not sure exactly what Time Warner Cable eventually wound up doing for us, but they appear to have us locked into some specific paths through the Carolina's now. They may just be be putting entries directly into the routing tables through our area...who knows for sure. I've notice when my route changes it is pretty much the same groups of IP's that get recycled again and again. We bombarded them with a TON of data back at the 2.0 launch, and again last fall. This may have led to them adding entries to direct us through some specific gateways for getting to SE's network in Montreal. It's sort of an old-school way of routing. There are ways to over-ride the default routing with static paths in the routing tables. For instance, they could add entries in routers in Charlotte, NC to send all packets destined for SE's Canadian IP's (or Ormuco's network in Montreal) to always go through a specific router that would direct that traffic differently than all the other traffic going through Charlotte and up the East Coast. It can be some serious micro-managing of the network to do this, so getting it done can be tricky--but it is possible if you have people with the resources to implement the changes.

And that is the problem in some of these situations. The people you typically first talk to on the phone can't look into these options, much less make the changes. You have to get moved to higher levels of support/authority to get to someone that can properly investigate and recommend these types of changes. But yes, it should be possible to get them to assign alternate routing somehow...but you need the right people involved to make it happen. The industry standard for that level of support is Tier3, but some smaller companies may simply refer to them as Engineering or their Network Administrators. I've found I fair better getting to more advanced people when I use the online support options (email and such). Using phone support can get quite frustrating... constantly on hold, switching you back and forth to different people. Sometimes it seems like they do it on purpose in the hope you will give up trying.

If you do manage to get someone higher up that is actually interested in trying to do something about it, they would need to know the main block of addresses for Montreal, which is 199.91.189.xxx. There are other addresses we also hit periodically in Japan (mostly during log-in and initial loading of the game), but for the gameplay portion on the Canadian servers, they fall in that 199.91.189.1 /24 subnet.