Welcome to the high end rigs crap performance club. Unless your PC is super overkill for this game it just doesn't run stable. It could be a few things, DX9 not 11, Drivers not optimized ( which they aren't really tbh ), Game not utilizing resources available fully.

I would direct you to this thread where we are having the same discussion: http://forum.square-enix.com/ffxiv/t...3-fps-in-towns.