Vote Abandon is the better way. Mostly you get a healer and he drops, cause no tank and vise versa.
And there is another reason who speaks for Vote Abandon:
It seems if all people just leaves a duty, the duty is still on the server for a couple of time, so if you join in progress, you can get into a empty duty.
So it seems like the vote abandon would close the duty, so no one can join it in progress.

Correct me if i'm wrong, but i got to much empty duties in the last months and i wonder how a empty duty can exist.