I think the issue may be toggling. It is a bit tough for me...I keep a potion and ether on my healing bar in case I need em...I also have my DoTs and stun...rest I have heals. It's hard to pick which 16 to have up for the controller and which to have to the backup....I turn off the menu bar when my weapon is drawn so I just have 2 bars.
Do you keep swiftcast on main bar or second bar. (second bar is protect, raise, and things not used too often.)