I'd say allow one withdrawal without penalty (to allow for either technical glitches or unavoidable RL issues), but if within half an hour someone withdraws from two groups after they're formed, then they get a 20 minute wait before they can queue again.

Oh, and withdrawals before the DF groups you up with other people don't count towards this. Sometimes I'll start in DF, then before it finds a group for me I'll hear from a FC member that they'll join. I think that's a valid reason for withdrawing in order to group with them and re-queue with both of us. It shouldn't be penalized, provided that my original time through DF had not yet paired me up with a group.