I had the same issue before heaven swarm, also the pet always took days to cast his ultimate or contagion.

Now it is really faster and does not fail often, but it still fails some time.