Agreed, OP. It is annoying, but it might be less so if macros were actually usable. As they are now, they're mostly a detriment since they remove the ability queue. Because of that, they can't be used to remove clutter unless you want to impede yourself.

I also believe that if CC abilities (stuns, silence, and the like) didn't have damage attached to them, they wouldn't be used for more than their utility, and that change alone would help make the skills feel less bloated since you wouldn't be using them for their free damage (SE barely makes fights that utilize them as it is, sadly).

I prefer complex bosses to button bloat, personally.