Having played with both, I perfer keyboard and mouse hand over fist.
That having been said, using a controller is intuitive, straight forward, easy and quick. There is no detriment outside of socialization with the controller taken a little time to adjust.
Oh, and they altered quest handins with the keyboard and mouse, right click the slot for the quest hand in, it'll pop up the item that goes there.