My Mudra cast times differ greatly depending on where I am, and the time of day. It's odd.
If I'm at my personal house, I can cast them lightning fast. However at peak times in Coil or something...not so much. It's very disconcerting to have them change all the time.

I don't think there are many tips to give, sadly. I would really like this to be changed however. It seems so...finicky.

For keybinds, I have mine set to Q, E, R, and activated with F. Works great for me.