It completely defeats the point of Iaijutsu to have it be instantaneous. This sort of iconic Samurai cinematography is exactly what they're aiming for, down to the tension-filled metallic hum before you unsheathe the blade. It's a bit like the dramatic pause during a western 'quick draw'.

Also worth noting that Iaijutsu has a massive 6 yalm range. This means that if you are actually forced out of hitbox by a PBAoE, it also doubles up as a pretty good uptime tool.