Fact is, this game's graphical engine is finicky and does things in the least optimized way possible, so it often draws upon massive amounts of power to do half of what it could.

That said, the benchmark is also a huge liar and spits out incorrect values, do not believe in it. Just a quick test and raise quality progressively to be sure you don't fry your board.

Edit: That said, certain configurations in the game's graphical engine are ALSO liars and do not work, some will drastically icnrease performance if you switch from software to driver mode, like Anti-Aliasing.
I turned off XIV's AA and activated my driver-powered AA, and good god, now even Ul'Dah loads with decent speed.