The only times i vote kick is from ARK/Disconnects or is someone is being a complete and utter nuisance to the entire party.
Most recent was a THM who refused to use anything other than Fire 1 throughout the whole dungeon