Once Vote Abandon becomes available, Leaving Party shouldn't apply the debuff.
That kinda defeats the purpose of vote abandon to begin with.