Stealing this from another thread, but this is apparently the solution, as it worked for me as well:
Quote Originally Posted by Hirakun88 View Post
try deleting any existing FFXIV Benchmark folders in your 'My Games' Folder, i was getting this same issue and was able to fix it just now using this method.