Unfortunately with the way it is set up now, anyone can re-join any duty whereby they were dismissed, by using "in progress" as an option. It's just less likely in the smaller instances where so many are running at once, but I was once in a particularly terrible Ultima's Bane run where someone rejoined after being booted for pulling the boss before buffs, then dying, because they were upset that they could not a get a one-shot clear through DF at 2 AM on a weeknight. No one else was queuing up to fill their DPS slot fast enough before they could re-join, and in the end we had to call it quits thanks to them.

Yes, we reported them and moved on, but in the end this could have been prevented through some sort of instance-based lockout, or a cooldown on that player's end as a result of being dismissed several times in a short time.