This issue--massive performance drops while panning the camera when Occlusion Culling is enabled--has been in the game at least since ARR was released (I did not play the beta, but I assume it was in there as well) and has yet to be addressed. Worse still is the fact it seems to be either ignored or swept under the rug by random forum trolls who clearly have no idea what they are talking about.
I took the liberty of taking four comparison screenshots which literally sum up entire paragraphs of the undeniable existence of the issue and the obvious cause. To those who don't want to bother examining the screenshots, I will sum it up it lay-man's terms: occlusion culling is a common technique used in games whereas objects/characters/models are temporarily ceased to be rendered when they are occluded by some form of geometry, such as a wall; this is a fairly common feature as most games would be unplayable/lagtastic without it. FFXIV has this feature as optional and, to put it bluntly, the feature basically stops working when the player rotates/moves his or her camera even a millimeter.
Again, before anyone has some troll comment, please, look at the screenshots and pay attention to the occlusion culling option in the lower left and the framerate display in the lower right or even the integrated one in the options pane.
Occlusion culling enabled while standing still and not moving the camera--63fps, looks good... (or about 15.8ms of frame draw time & 80% GPU usage for the tech savvy)
http://i.imgur.com/4oSGCdZ.jpg
Occlusion culling enabled while moving the camera a few centimeters--dropped to 43fps, a 46.5% performance hit (about 25ms of measured frame draw time & GPU usage at 55%, whereas it should be around 23ms, so there is an immediate pipeline problem here)
http://i.imgur.com/t7VX3vx.jpg
Now, notice what happens when I turn occlusion culling off while the camera is not moving.
http://i.imgur.com/T9gxnMF.jpg
Interesting... getting the exact same framerate and frame draw time as before with occlusion culling bugging out
To put the nail in the coffin, notice how there is literally zero performance hit with occlusion culling off and moving the camera
http://i.imgur.com/RNdMFwv.jpg
As much variables as possible were accounted for and no screenshots were altered in any way.
I have tested this issue with the 331.65 drivers (latest), 331.58, 327.23, 320.49, and the 320.18 versions (cannot go back farther on a GTX 770) and the performance drop remains consistent throughout--though the latest three drivers have consistently better performance in GPU bound areas.
Finally, before someone claims it is a problem on my end, I am using the following which obtains a 12300~ performance score on maximum @ 1920x1080:
4.0GHz (locked) i7-930 (a 4.5GHZ 4770k will be about 25-30% faster in a 4-core application)
1215MHz GTX 770
6GB RAM @ 1333MHz (total RAM usage never exceeds 3GB while running FFXIV)
Windows 7 Pro 64bit
Samsung 840 Pro SSD (game drive)
screenshot or it didn't happen: http://i.imgur.com/K4xqV6b.jpg
I have also tried even single possible idea anyone can think of, so please don't recommend any run-of-the-mill ideas; these would include things such as hyper-threading on/off, vsync on/off, updating drivers, running in windowed, lowering settings (almost all settings are GPU related and have almost no effect in this situation as it is strictly CPU related), putting it on an ssd (it is), re-installing windows, turning off background applications, turning off SweetFX, turning off MSI Afterburner, stop overclocking, unparking cores, turning on/off speedstep, turning on/off C1E support... you get the idea.