I've been having issues with this for months as well, wanna say since patch 4.1.
My ISP is Comcast and I live on the east coast. The connection issues and lag starts happening around 8pm est and lasts until around 1-2am for me.
Code:
Tracing route to 204.2.229.97 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  10.0.0.1
  2    13 ms    16 ms    10 ms  96.120.66.161
  3    10 ms    13 ms    13 ms  68.85.187.237
  4    13 ms    13 ms    15 ms  be-22-sur02.lowell.ma.boston.comcast.net [69.139.221.246]
  5    17 ms    15 ms    14 ms  be-21-ar01.needham.ma.boston.comcast.net [68.85.106.45]
  6    25 ms    22 ms    24 ms  be-7015-cr02.newyork.ny.ibone.comcast.net [68.86.90.217]
  7    20 ms    26 ms    18 ms  be-10381-pe02.111eighthave.ny.ibone.comcast.net [68.86.86.250]
  8    47 ms    54 ms    53 ms  50.242.151.70
  9   190 ms   178 ms   168 ms  ae-9.r07.nycmny01.us.bb.gin.ntt.net [129.250.3.93]
 10   114 ms   102 ms   102 ms  ae-2.r24.nycmny01.us.bb.gin.ntt.net [129.250.3.180]
 11    98 ms   193 ms   162 ms  ae-2.r20.sttlwa01.us.bb.gin.ntt.net [129.250.4.13]
 12   163 ms   106 ms   101 ms  ae-0.r21.sttlwa01.us.bb.gin.ntt.net [129.250.2.54]
 13   180 ms   194 ms   186 ms  ae-3.r23.snjsca04.us.bb.gin.ntt.net [129.250.3.124]
 14   211 ms     *      232 ms  ae-41.r02.snjsca04.us.bb.gin.ntt.net [129.250.6.119]
 15   175 ms   187 ms   176 ms  ae-3.r00.scrmca02.us.bb.gin.ntt.net [129.250.7.11]
 16     *      223 ms   209 ms  xe-0-1-0-1-1.r00.scrmca02.us.ce.gin.ntt.net [129.250.195.46]
 17     *      208 ms     *     204.2.229.234
 18   213 ms   224 ms   221 ms  204.2.229.97

Trace complete.