That is a really strange problem!

Does it disconnect you if you jump when spellcasting?

Try pinging the servers with this guide:
http://www.ffxivguild.com/how-to-tes...agpinglatency/

Do you get any lost packets? What is your ping?