Rather then a penalty for not accepting, a better solution would be to keep the same group that was queued up before the cancellation with the exception of those that backed out or did not respond by the time limit.