A vote kick system certainly does need to be implemented. It is much more unlikely that a group of people will abuse something than a single person. That being said, any system needs to be policed - if someone feels they were vote kicked without just cause they should be able to report it. If someone abuses either the vote kick or report system then take these tools away from them until they learn to behave themselves.

It is much easier to police an active system than it is to do something about some guy who goes afk and could simply claim lag/dc etc...