Without a penalty, people will leave immediately, thus wasting everyone else's time. That being said, I do think the penalty should apply to more people; upwards of three in eight man content. That way the remaining five aren't held hostage if queues aren't filling up.