I had the same problem when I started playing whm. But then I got used to tap it only once.
Just remember that the effect comes after the previous cast or animation is finished. If you deactivate it while casting, let's say holy, then the cleric stance icon will disappear after holy is complete or canceled.
Just make sure you don't tap it more than once.