I use a controller and Main WHM, and typically am faster at buffing/healing than other whm's I'm around...BUUUUTTT

1. I reconfigured my Controller to be what works quickest and best for me.
2. I have anally laid-out my action bars so that I have only 1-2 clicks between most skills, especially the most used or combo'd ones.
3. I use my targeting filters
4. As whm, I find it best to hard target the person who's going to get the most damage so you don't need to reclick them...then if someone else needs attention I just have to click up or down in the party list (you can do this from a gamepad) to heal them, but after that, my target is automatically still the tank or whoever is most likely to need my constant attention...meaning I don't need to re-target or move my mouse back to them.

Basically, this is a preference thing...using a controller requires a little more pre-planning and setup to get the smooth, quick gameplay you are after, but it works. But for folks use to gaming mouses/keyboards etc, the more 'traditional' PC routes probably work better. This is solidly a style thing.