You could do other things in-between waiting to recast more spells. RDM has quite a few insta-cast actions that can easily keep you busy, Fleche, Contre Sixte, you could even throw in Acceleration or Manafication depending on your rotation. The main good thing about the casts of RDM is that it can be very mobile, whilst BLM on the other hand is your constant caster acting as a destructive turret. Honestly every caster has a little downtime between spells, but good players find ways to fill those gaps.