It has to do with delay, sadly, and we can't do anything about it. That's SE's fault. And yes, the inconsistency is extremely annoying and forces us to lose DPS, which I am not a big fan of.
A tip is to go to your FC house and do your mudras there, you will notice it flows MUCH better in there.