Weird lag/graphics glitch

I seem to have an issue (particularly during the FATE near Horizon) where I get into a huge lag problem and it takes minutes and minutes for my actions to catch up. My net connection is ok but I can't keep up with the current actions and if I leave the area it will take minutes for the actions to catch up. I typed leave and left a party but it took 4 mins before the actions had caught up. Once I lag out that badly, I then continue to do so. My character moves according to friends on voice chat but for example I 'returned' and on my screen I was still hovering in mid air but my friends registered me as having made it to Limsa.

Help!