I used to use a GTX 460 and never got this. Instead, I would get NVidia display driver crashes and recoveries (also called TDR crashes).

As soon as I switched to an R9 270x, I never got display driver resets, but I instead started getting the 1-2 FPS for several seconds, then either a client crash or a client freeze.

After observing the problem for months, I believe the underlying cause is the same problem in the FFXIV client. On NVidia, display reset. On AMD, client freeze/crash.