I switch to controller for certain types of activity, such as long stretches of gathering or when I make certain jumps for the exploration journal. But generally I have trouble with precise targeting using a controller. The auto-switch target function is pretty scatterbrained in this game (targeting a passive enemy out of range when there are two enemies smacking me in the face) and ground targeting attacks and pet movement just aren't quick enough for me.

Not to mention I love having tons of shortcuts and rows of toolbars so I can do a dozen different things without opening a menu. One toolbar for instant job changes, one toolbar for professions, one toolbar for mounts/pet/potions/companion skills, etc. I think the controller crossbar barely has enough room for combat skills.