I suggest getting a Gigabyte Geforce GTX 660 Windforce. It's affordable, it's powerful but not excessively so in a way that your CPU will bottleneck it too much, and it's not too power-hungry. If your power supply is a 450 or higher with a single connector it should be ok.

PS: Yes, upgrading your GPU WILL very sizably upgrade your performance. Whatever people say, FFXIV: ARR's engine is very GPU intensive and very little CPU intensive. There may be some bottlenecking but not nearly enough to snuff the power of a good GPU.

Adding one gigabyte of ram and bringing your system to 4 will also help, since you run a 64 bit OS.