often times a lot of people want to leave but fear the queue cooldown so don't. Being the one that gets the punishment really is the only thing keeping people in line after all
As soon as the person that is so fed up they don't care at all leaves they are given the opening to do so as well
then if the party dwindles enough other people that were fine are now shaky that so many spots need to be filled and leave as well

it's not the most complicated thing really