The party leader doesn't have the ability to single handedly kick people in dutyfinder stuff. You were presumably voted out, you can't see votes being held against you.