Lack of SE Response to Lag

[Marishi-Ten]

I've been troubleshooting since about a week after 2.1 released. My ISP (AT&T U-verse) claims no other users have informed customer care of having issues with this game. I've shown them trace routes detailing most issues going through nodes owned by Cogentco. AT&T U-verse says, if there aren't any other issues with my network (true), there's nothing they can do to help me. SE NA chat support has been helpful in teaching me how to troubleshoot. I didn't know that port forwarding, DMZ mode, or open DNS were real words prior to December. Learning is fun. Google DNS did have some positive results...not enough to make current end game content completely playable, but it's something.

Cogentco is another backbone that has a horrible reputation. I'm pretty sure they let fires rage at their nodes they suck so bad.

Yeah, I hate calling ISP's. 90% of the people you talk to have no idea what's going on. If I have to talk to my ISP, I will bomb the CS rep with jargon and raw information till they send me to T2 where I'll do the same till I get to Engineering. They can change (called prioritization) your route path to avoid crappy backbones. No one likes to do it and they DEFINITELY don't talk about it (they don't want to be bombed with requests and put load on another backbone that's probably reserved for commercial/VIP tier clients). Essentially, you need to get into contact with at least entry level Engineering Support. There, they should know enough to understand what you're saying and put the W/O in.

[Billie21]

More or less. You can test this yourself by pinging your servers IP address. Famfrit's IP is 199.91.189.39 - http://arrstatus.com/

Open your command prompt and type (without quotations) "tracert 199.91.189.39" and hit enter. Watch for asterisks and high latency values. You will have to run this multiple times as the bandwidth ebbs and flows. You should be able to tell if the issue is on Square (asterisks are after Montreal) or on the node/ISP/backbone (before the Montreal hop).

I've done tracert before. My latency always spikes (more than doubles) from the last hop before SE. Though the numbers aren't that high, tbh (nothing like the 200-300+ ms aussie players often describe).

10 46 ms 45 ms 45 ms be2283.ccr21.lpl01.atlas.cogentco.com [130.117.49.34]
11 116 ms 114 ms 115 ms be2384.ccr21.ymq02.atlas.cogentco.com [154.54.44.137]
12 117 ms 116 ms 116 ms 38.122.42.34
13 116 ms 172 ms 116 ms 192.34.76.10
14 117 ms 119 ms 116 ms 199.91.189.242
15 116 ms 116 ms 116 ms 199.91.189.39

Blizzard did what? Sign a service agreement with multiple ISP's for direct drops to the node over a pure fiber connection with their own racks where the ISP probably had the same thing done to a major node in the area? I'm pretty sure they did. I can't tell you 100% that they did, but if you can get me root access into their terminal(s) I can tell you.


I wouldn't call it an assumption, but rather a hypothesis. As I said before, though I don't work in the gaming industry, I do work as an IT admin and no matter what the end application is, data is data and infrastructure is typically universal. I try to put myself in the companies shoes and I have dealt with backbones and ISP's (they are difficult to deal with to say the least).

TATA isn't even a T1 backbone in the US. They are T3 at best. Have you asked your ISP for a re route? They may be more inclined to re route around a T3 since they normally don't pull a whole lot of weight.

I'm not invalidating your concerns, they are obvious to see. I just don't know what people expect Square to actually do. It's lose-lose for them:

1.) They can come out and say there is latency that is impacting performance but then people see them taking responsibility and demand THEY fix it (the average Joe who thinks of the internet as a singular entity) making them look bad and instigating bad press hurting sales and generating churn.

2.) They can say there is latency impacting performance and it looks like the issue lies with another company and to call them (their service contract may bar them from doing this and if they start pointing fingers without being 100% sure, they WILL be sued for libel).

3.) They can simply not say anything. Yeah, people will be upset and cause an uproar but at the end of the day, they are able to preserve their name as a business, not be held accountable for issues not their own, and avoid law suits.

Their admins in Montreal are probably taxed as it is and after the whole 3102/90000 thing I would imagine someone got the axe for that (not their fault, but someone's gotta be accountable for stuff like that). I also imagine that they are on the phone with their ISP/backbone/providers daily to try to route around or figure another solution (there really isn't one). Also remember that their admins are responsible for maintaining the integrity of their clusters, monitoring the hardware for changes/driver conflicts/temp overloads/etc. while running private server testing for future sprints, monitoring software stability and clearing VRAM cache (this one blows my mind. IF YOU HAVE MEMORY LEAKS THAT BAD, YOUR CODING NEEDS REWORK OR YOU HAVE BAD HARDWARE). They don't rent their racks either which means it's all on Square to manage and monitor their internal hardware/software and from what I recall, they aren't doing so hot financially and we all know that when times get even remotely tough for a company, they start the layoffs (especially for service/IT/Software/Engineering companies).

I'm just as irritated as you at the latency issues and that there is no visible movement, but I have experience in the field and I know that Square's hands are pretty much tied in the situation (from a maintenance and business standpoint) so I have to either VPN (which I do BOI > Vancouver > Montreal direct tunnels) or take it in stride and file bugs with the backbone/ISP till it gets fixed. Though I'm quoting your posts, my responses are more for general information and not directly contesting your points.

You're basically confirming I was right to unsub.


There is a difference between "can't" and "won't", a difference between "impossible" and "can't/won't afford".

Also, you can only speculate. I'm not saying you're incompetent (I wouldn't know), but the only one that could confirm this is SE.
When they reduced the position checks from 0.3 to 0.1 in specific dungeons, I was hopefull that they would be doing something in the future, and that's the only reason why I didn't unsub at the time. Since then, they have outragiously dodge the question, which is why I have unsubed after all. Just to illustrate how communication can actually make a difference between money flowing in or not.

Even if all you're saying is true, then they could(should) have designed the game accordingly, not basing difficulty solely on twitch reaction dodging-or-dying mechanisms that cannot tolerate the slightest delay.


Again, customers aren't all IT techs. They don't want to have to tweak their PC, call their ISP, beg for favors or pay additional services to be able to play. If the game doesn't work, they stop paying. Or if they like it enough, they complain. Then stop paying when they are ignored/ridiculed (I'm still talking about SE, not other players... again, communication can do wonders).

Maybe it's finacially more interesting for SE to ignore this problem, as you are guessing (I had guessed as much when Yoshida started dodging the question when interviewed).
I just wish it didn't make me feel cheated and wishing I hadn't given my money to people who just weren't interested (financially or otherwise) in delivering.

If they are actually trying to negociate with other companies, improving their network, or whatever else (even just investigating, even if they aren't sure they will be able to do anything), saying so to the players might have been a good move. Personally, I'd have keep my sub running, and bid my time if that was the case. I like this game as much.
That's actually what I did before Yoshida started saying they couldn't design hard content for 3 digit latency. That's a great example of terrible communication.

Btw, I thank you for keeping a cool head and trying to be constructive.

[Marishi-Ten]

I'm sorry but who do we blame ummm who makes the game who are we paying to provide us the game and the servers to play on. SE so its there fault they should be offering ppl a stable and reliable service for there subscription they receive ..i care little about the tech side of it and all the tech bullshit ppl on this thread spin i pay to play a mmo thats not offering me a reliable service and i pay SE so they should fix it!!

This is the mentality I'm trying to get people to avoid. This kind of thought process doesn't help anyone. This is the EXACT reason why Square doesn't say anything.

You pay Square to rent a small portion of their servers with others. Read your terms of service. They don't promise you stable connectivity and have no obligation to provide any kind of networking upgrades of any kind.

I'm not spinning anything. If you took time to educate yourself and understand the situation, you would see that the root cause of the problem isn't with Square (data-wise).

I'll break down your options for you:

1.) Stop paying
2.) Get a VPN (I recommend WTFast personally. The tunnel to tunnel connection is pretty sweet)
3.) Move to another area or get another ISP (if available)
4.) Wait it out

You could even remove 3 and 4 if you wanted which leaves you with 1 and 2. I don't have any skin in the game so I don't really care what you choose to do.

[obscenelocust]

Cogentco is another backbone that has a horrible reputation. I'm pretty sure they let fires rage at their nodes they suck so bad.

Yeah, I hate calling ISP's. 90% of the people you talk to have no idea what's going on. If I have to talk to my ISP, I will bomb the CS rep with jargon and raw information till they send me to T2 where I'll do the same till I get to Engineering. They can change (called prioritization) your route path to avoid crappy backbones.

I LOLd so hard at the fires bit. Ty for that. I'm pretty sure I'll never be able to talk to Engineering. Most CS reps at U-verse want me to pay $15 a month extra to go through Connectech, claiming that's the highest tier of customer support they offer. Uhm...no. Unfortunately, I can unsub from this game a lot easier than getting out of a contract with AT&T. Bigger unfortunate is that the only other broadband ISP in my area is notoriously worse than AT&T.

[Marishi-Ten]

snip

You actually have pretty impressive latency values. 45ms on a remote node is insane good. The double down would be due to the switch over from a central node (hop 10) to a local hub (hop 11) where hop 12 is internal route local to SE. Central nodes carry alot more traffic so when it comes time to switch to the hub it adds on (equipment is probably better at the node. It sucks but it happens. Hopefully Congentco can figure out networking and do, you know, their jobs).

You're right and I don't mean to sound negative but people get tunnel vision and focus on one thing. I try to act as if I were in the companies shoes. If I were the IT admin for Square, I would be taking orders from my lead who gets them from the manager who gets them from the producer who gets them from the shareholders.

Sadly, this is the modern age business model. until the whole thing falls apart nothing will change (if it ever will. You can't stop greed). I do believe however that the little guys (the employees on the ground that build the framework, run the day to day systems and monitoring) have best intentions in mind and are trying to do their best (within their scope and sprint cycles). I also believe that the admins have and are in contact with the node providers trying to optimize connections and following up on bugs (sucks that everything is so fragmented. Makes it really hard for any progress to actually be made). There is just red tape everywhere.

Square is publicly traded and any statement made is taken as gospel and dissected. Square has to tread incredibly thin ice as it shows the company as incompetent if they say "hey things are messed up, but we are trying our best to sort it out". It reads as "we aren't smart enough to get things under control and we don't know when or if it will be fixed, but hang in there". Marketing/Legal won't let that get out. If Square says "<insert ISP here> looks to be having issues routing traffic, but we are working on it" that reads as "<insert ISP here> is stupid and it's their fault things are messed up. Call them as they are the ones being difficult". Marketing/Legal won't let that out as even if they have the data that supports it, they are wide open for a huge Libel suit. The lesser of evils is to say nothing at all

I'm willing to say with 90% confidence, Square is aware of the backbones having problems and likely are trying to get it taken care of (a phone call a day to piss off the backbone enough into action, subtle threats about re routes causing lost revenue for the BB, etc.) Also remember, there is a language barrier and that the Montreal office is a satellite office (HQ has to read through the data, determine their findings, have a meeting, determine what to do, have another meeting, bring in Joe from Engineering for no apparent reason, draft a resolution workflow, have a meeting, train HQ on the workflow, get the workflow signed off for approval, translate the workflow, send it to the satellite office, satellite office trains on the workflow, satellite office runs workflow into production).

I'm just a random person on the internet for good or bad. Square doesn't pay me and you can take everything at face value. I might be wrong in thinking all of this. All I can really do is share my experiences working in the IT world, how it works, and what the challenges are.

Hopefully it sheds some light on the situation and educates others.

[Marishi-Ten]

I LOLd so hard at the fires bit. Ty for that. I'm pretty sure I'll never be able to talk to Engineering. Most CS reps at U-verse want me to pay $15 a month extra to go through Connectech, claiming that's the highest tier of customer support they offer. Uhm...no. Unfortunately, I can unsub from this game a lot easier than getting out of a contract with AT&T. Bigger unfortunate is that the only other broadband ISP in my area is notoriously worse than AT&T.

I've dealt with AT&T over their network routes before (lolMaBell freaking monopoly to this day) and they are a pain to convince it's their bad (excel sheets GB's large. traceroute -n <IP> for DAYS). You can ask them to open up a trouble ticket and send it to their applications support team (Engineering) for back end review. Make sure the CS rep notates exactly what's going on, what you want, what your expectations are, your contact information, and data to corroborate your issue if need be.

When I have to call Rackspace (UGHHHHHHHHH you think Square is bad call Rackspace. Makes Square look like UNICEF) I usually have all the data compiled and available to be reviewed and sent. I'm also firm about what I'm requesting but not aggressive. Having a cool CS rep goes a long way in getting issues resolved. It's been my experience that if you're pissed and it flows over to the CS rep, odds are that ticket will get created under "Administrative" "Historical Reference" and closed out.

[Ricky]

I've said it before and I'll say it again, all pingzapper/WTfast is, is a VPN service, it allows your traffic to be routed via a private network as opposed to the public internet, it's still connecting to the SAME routers and servers on SE's end, therefore if this increases your ping and actually lowers latency, then the issue isn't specifically on SE's end, the problem is a, the internet, millions of people using the infrastructure, so SE could slightly bump up the cast on AoEs for this, or b, your ISP is using a sub-par route to get to the datacenter.

People saying "SE YOUR NETWORK SUCKS" then says how a VPN makes it much more playable makes me laugh every time.

so many commas

[snakey77]

This is the mentality I'm trying to get people to avoid. This kind of thought process doesn't help anyone. This is the EXACT reason why Square doesn't say anything.

You pay Square to rent a small portion of their servers with others. Read your terms of service. They don't promise you stable connectivity and have no obligation to provide any kind of networking upgrades of any kind.

I'm not spinning anything. If you took time to educate yourself and understand the situation, you would see that the root cause of the problem isn't with Square (data-wise).

I'll break down your options for you:

1.) Stop paying
2.) Get a VPN (I recommend WTFast personally. The tunnel to tunnel connection is pretty sweet)
3.) Move to another area or get another ISP (if available)
4.) Wait it out

lol what mentality are you looking for ??.. i simply said i don't give two shits about the tech it just should be fixed and simple logic is its se problem .. frankly your options are as ridiculous as you overall post of which wasn't directed at you .. stop play yeah okay not voice my opinion .. get a vpn I'm on ps3 .. so now I'm to buy a pc to play and then pay for more shit plus a sub .. hmm needless to say these forums are getting worst it sounds to me you can't make up your mind to either kiss there arse or fuck it

[Nezacant]

For the first time since the 2.1 patch I do not have any lag tonight.

[obscenelocust]

snip

Ugh, how dare you show me a constructive way to attempt to get through to MaBell when I had already given up hope!