If using the updated 2.1 launcher, you can find the option to backup/restore at the Launcher > Config > Backup Tool.

Or you can do it manually by going to: "C:\Users\<username>\Documents\My Games\FINAL FANTASY XIV - A Realm Reborn". Config (Character data, screenshots, Macros and HUD etc) are stored here by default.

Game folder is at the location where you specified during installation; I changed mine so for me it is at: "C:\Games\SquareEnix\FINAL FANTASY XIV - A Realm Reborn". Like most other games these days, keeping safe of this folder to another partition or hard drive will enable you to play without major reinstall from the gold copy (initial released without any patch).