Strange freezes, crashes and lag

Hi.
I came back to play this game again a while ago, and I was pleasantly surprised to find it running rather well compared to how I remembered it. Went ahead and leveled to around 45 and it all went very good. Then around the time of 1.22a I started getting major issues. I cannot remember if it was before or after the patch, but I'm quite sure it's after.
Aside from the general actionbar latency many seems to experiance I got some severe issues that makes the game unplayable for me.
Mostly it acts like lag, everyone stops moving and I'll be able to move my character around (unless I try to mount up) but I won't be able to start attacking something, open my map, options beside the side-menu, and so on. Sometimes it freezes completely. And sometimes all animations stops working but the game still registers my attacks and such, though no recast timers combined with no animations makes it really tough to finish a fight properly. Usually if I wait for about 5-10 minutes the game will go back to normal, but these issues occur almost constantly, more downtime than playtime so to speak. And at times it won't come back, just goes "Not Responding" forever. When these issues happen other programs are very prone to freezing or crashing at the same time. Google Chrome tends to say something like "loading cache...", while Skype just dies.
I usually don't get kicked off the server or anything, no error messages, only when I did ctrl-alt-del the computer frooze for a few minutes then when it all came back I got the obvious DirectX device error. I've tried almost everything I can think of now, and nothing seems out of order, GPU (temps, usage), CPU or memory, no other applications seems to have any issues. Even my flaky connection's been great (I managed to do a check once while the issues were happening and my connection was perfectly normal).
So far I've tried all different video settings, enabling and disabling, going up and down in quality, 30 FPS as well as 60, full screen and windowed, both without any background programs. I've even completely reinstalled and repatched my game (removed all old files, documents and did a proper registry sweep)...
My system is a Win 7 64-bit SP1, i7-2600k at 3.40 GHz with 8 GB RAM and a nVidia GTX 560 Ti card by MSI.

Many thanks for reading.

Slight update, I noticed that during the latest freeze my GPU usage dropped to around 0% for the duration of the freeze.

You are not alone. I spent a week figuring out my setting on a new card. Then I played for a week smooth as can be in the field. Now what appears to be lag. UI latency is bad and the stuttering in the field where it was capped at 60 when solo really grinds my gears.

i felt that yesterday some stuttering in the graphics.
while walking in gridania or during the story events.

I'm having same problems. Also, when I "ride the aethryte" from anywhere to the Carline Canopy in Gridania, it takes about 20 secs for myself to appear on the screen. I can move and everything, but I'm invisible. It then takes another 50 seconds for the NPCs in the Adventurer's Guild to appear.

This all started after the most recent update.