Why not have it so that a Vote Abandon after 5 minutes and before 15 minutes has to be a super-majority (3/4 of the party, or rather 6/7 because the initiator doesn't vote) in a full party, and unanimous in a light party? That way if everyone agrees that it's hopeless they can all vote to just get out, but still prevent some jerks from trying to abuse it. (Just an idea, I have no idea whether it would actually work or not.)