Quote Originally Posted by Raist View Post
Actually, if you look more closely, you will find the typical signs of congestion issues occurring earlier in the routes that may be at fault that need to be looked into first. For instance, look at the wide variance at Level3's hop shortly after coming around Chicago:

10 116 ms 93 ms 70 ms ae-10-10.car2.Montreal2.Level3.net [4.69.153.86]

That's a swing of over 65% variance in a short time frame. If this is occurring regularly, it is a sign of pending (or even actual) overload either at the IPX or coming through Level3's segments. Such an issue compounds itself the further you go down the route. The fact that that hop further down is having a much more consistent response time is actually encouraging--it means it is doing a better job of maintaining traffic flow. The fact the response times are markedly high could very well just be because the shaping policies in effect have put the ICMP ECHO request on a lower priority, which means it is working as intended--slowing down less important traffic to reserve capacity for higher priority traffic. This is done to try to maintain more consistent flow--which clearly is not happening at the other hop.

Some other examples from one of the linked threads:

3 25 ms 8 ms 11 ms G0-6-4-6.WASHDC-LCR-22.verizon-gni.net [130.81.212.144]

4 33 ms 11 ms 14 ms xe-1-1-1-0.PHIL-BB-RTR2.verizon-gni.net [130.81.209.18]

so-9-0-2-0.RES-BB-RTR1.verizon-gni.net 0 150 150 7 18 100 9
(best time of 7, average of 18, worst of 100)

And this one is a REAL doozy:
Code:
Host	%	Sent	Recv	Best	Avrg	Wrst	Last
Wireless_Broadband_Router.home	0	500	500	0	0	1	0
L100.BLTMMD-VFTTP-52.verizon-gni.net	0	500	500	5	7	114	7
G5-0-352.BLTMMD-LCR-03.verizon-gni.net	0	500	500	5	7	118	7
so-9-0-2-0.RES-BB-RTR1.verizon-gni.net	0	500	500	7	18	176	99
0.ae5.XL1.IAD8.ALTER.NET	3	457	446	7	10	48	42
0.xe-10-1-1.GW9.IAD8.ALTER.NET	0	500	500	7	11	123	12
tinet-gw.customer.alter.net	5	421	401	15	19	97	17
xe-1-0-0.mtl10.ip4.tinet.net	0	500	500	25	37	184	37
ormuco-gw.ip4.tinet.net	0	500	500	42	45	148	43
192.34.76.2	0	500	500	42	46	147	45
199.91.189.234	1	496	495	40	45	170	41
199.91.189.30	0	500	500	40	43	147	41
In that sample, it's ALL on Verizon and their Alter.net subsidiary before it even get to TI-Net, which is another offender at times as well.

The root of this is indeed (in the majority of cases) an issue of flow control, compounded by shoddy/outdated peering agreements between our last mile ISP's and their routing partners. It has been discussed over and over on various forums for years (not just XIV... I remember it going back to the early days of WoW, Diablo, and even Freespace: Descent). Articles have been published and briefs/complaints filed with the regulatory commissions trying to get the issue addressed. One such blog post was put up by Level 3 this past year (you can find more on the topic by simply googling "level3.net chicken", or follow the links that crop up with that blog post.) TATA also filed briefs on similar observations about 6 or 7 years ago. There was a big kerfuffle over Verizon, Comcast, and Netflix in more recent history. It's all bound to an issue with the ISP's cutting corners and trying to make more money. All the while fleecing us for all we are worth while providing us with sub-par service.

Need to hold your ISP's feet to the fire to address this. They have the resources to address the issue--need to get their Tier3 people involved so they can properly investigate the issues and push for the escalation to get them addressed.

It should be noted that a few months back, while working with my ISP's T3 support, we spotted an issue on the other side of Cogent's transit in a VLAN managed by Ormuco. It was routinely spiking upwards of 280ms from it's norm of around 100ms. We went straight to them and got it addressed without getting SE involved. This CAN and SHOULD be handled through your ISP, since you are paying them for the service of routing you across the internet, and THEIR routing policies are what is keeping you on these troublesome routes. If they cannot get the responsible party to address the problem directly, they have the option of changing you to alternate routing partners to try to find a cleaner route. To that end, TWC changes my route every few weeks trying to stay ahead of the congested hops. This is done entirely through my ISP's T3 support... no one else--I just email them with my reports, and they take care of it. Sometimes they catch before I do... kind of annoying when my modem bounces to re-align my channels when I am in game, but at least I am able to log right back in a few minutes later and continue playing with minimal issues.


Edit:
After eating dinner, I ran a trace from the back of the house (from laptop, on Wifi from about 30 feet away), and from here in South Carolina, at 7:30PM... trace is clean as it usually is. I don't experience this horrible lag you speak of, and I am going through that very same hop---but it is giving me mostly right around 60ms response times (rarely breaks 80 even under the busiest of times unless something is up along my route, which my ISP usually addresses pretty quickly now that I have gotten them to take action).

Code:
C:\Windows\System32>tracert 199.91.189.30

Tracing route to 199.91.189.30 over a maximum of 30 hops

  1     2 ms     1 ms    <1 ms  LPTSRV [10.10.100.1]
  2    37 ms    26 ms    28 ms  cpe-066-026-112-001.sc.res.rr.com [66.26.112.1]
  3    17 ms    24 ms    29 ms  cpe-024-031-198-005.sc.res.rr.com [24.31.198.5]
  4    13 ms    16 ms    14 ms  clmasoutheastmyr-rtr2.sc.rr.com [24.31.196.210]
  5    28 ms    25 ms    27 ms  be33.drhmncev01r.southeast.rr.com [24.93.64.180]
  6    32 ms    34 ms    35 ms  107.14.19.20
  7    33 ms    43 ms    33 ms  so-1-1-1.c1.buf00.tbone.rr.com [66.109.1.113]
  8    32 ms    31 ms    32 ms  ix-17-0.tcore2.AEQ-Ashburn.as6453.net [216.6.87.149]
  9    62 ms    64 ms    61 ms  if-11-2.tcore2.NJY-Newark.as6453.net [216.6.87.138]
 10    62 ms    76 ms    62 ms  if-4-4.tcore2.NYY-New-York.as6453.net [66.198.111.18]
 11    59 ms    60 ms    62 ms  if-12-6.tcore1.CT8-Chicago.as6453.net [216.6.99.46]
 12    58 ms    61 ms    60 ms  if-22-2.tcore2.CT8-Chicago.as6453.net [64.86.79.1]
 13    60 ms    69 ms    61 ms  if-3-2.tcore1.W6C-Montreal.as6453.net [66.198.96.45]
 14    59 ms    73 ms    61 ms  66.198.96.50
 15    62 ms    62 ms    64 ms  192.34.76.2
 16    58 ms    59 ms    60 ms  199.91.189.234
 17    62 ms    62 ms    59 ms  199.91.189.30

Trace complete.

Ping statistics for 199.91.189.30:
    Packets: Sent = 30, Received = 30, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 58ms, Maximum = 69ms, Average = 60ms
You've done some exellent work, thank you. As you said, there does not appear to be any lag... tonight at least. Part of what is frustrating is catching this in action. I'm hovering between 50-60 ms tongiht, which is outstanding compared to the normal laggy 150+ms