I haven't had any problems with it. Whenever I want to precisely position the camera the right-click+drag motion works fine for me. Is your framerate generally good otherwise? A poor framerate is going to make it sluggish.

Also, if you're a returning player. Did you go into FFXIV Config and turn "Hardware Mouse" on? If you're still using the software mouse that may add even more delay, but I'm not sure if it applies to the camera as well as the on-screen cursor.