You're confusing this with a slightly different issue... Party Finder and Duty Finder are 2 different features of the game. By using the Party Finder, OP would be creating a group before queuing up for a dungeon, meaning he would be the leader and well within TOS to kick who he sees fit between dungeon runs, and would not involve use of the Vote Kick feature at all.