The biggest issue you'll have I think is with performance. Your seek times will be faster on a USB Flash drive (I'm assuming you're going for a USB 3.0 one?) but I'm not sure how big the data files that needed to be loaded on the fly are.
List of some USB 3.0 Flash Drive Options
By the way, there is a way to solve the whole "looking for files" problem that Catapult is talking about. Look up the use of junctions. These are basically pointers that makes Windows THINK the files are in the original location, where in reality within the file system it points to somewhere else.
I would first suggest installing FFXIV to your usual drive so the installer makes the necessary registry changes to your system to tell it that FFXIV is installed. You can then copy over your My Documents\My Games\Final Fantasy XIV folder to the flash drive, then create a junction point where the My Games\Final Fantasy XIV folder originally was to point to a folder on the Flash Drive containing all of the files.
You then copy over the FFXIV installation located in C:\Program Files\SquareEnix\Final Fantasy XIV to your flash drive. Rename the old FFXIV folder to something like FFXIV.bak just in case things screw up. After copying it over, you make a junction point in C:\Program Files\SquareEnix called Final Fantasy XIV and point it to where the FFXIV install is on the flash drive.
When you launch FFXIV from the start menu and Windows goes looking for FFXIV, for all intents and purposes it will still think FFXIV is on the main drive. The only way you can tell it's not the original is that your junction point will have a little shortcut icon. However, when you open it, it will still look like you're looking at the folder as though you didn't move it.
I used junction points to move my FFXIV install from a mechanical hard drive to my SSD. Worked like a charm.