I normally find after 2nd or 3rd fail the person who initiated multiple vote abandons leaves. After this we get a new player and we clear the dungeon straight away.