I'm having this problem, too. Just tried to do a treasure hunt on SMN and got one where it spawns 1 boss + 2 waves of four adds. I got the first wave of adds and could not switch over to my pet hotbar to change its attack. I ended up having to Physick burn myself to stay alive. It got extremely ugly for a fight that should have been complete cake.