I know its the same issue with Cleric Stance for me, especially when it gets hectic turning it off and accidentally turn it right back on.