Happens to me too.
And whenever I bring it up on the forums I get the "use WTfast it's your ISP's fault you lag"
and I bought a month of WTfast to try it out and it's not doing anything at all, "connection improvement 5%" but still lagging like crazy.

During prime time I can't do Ex primals so I have to play on my alts doing low level dungeons where I can't cause wipes