I have ultimately felt like the Vote to kick feature has come up short in a lot of regards. Yet at the same time, I have seen it show a good light come from it, too.
First and foremost, the issue with the system is that it requires a 50% vote, and not a 50%< vote (I.E., it requires 2/4, not 3/4 or 5/8.) This is a big problem because it means that 9/10, people will kick for no reason whatsoever (Moreso just because they click yes without even really thinking of it. I've seen a number of people get kicked without a word said, not because they did anything wrong - but just because someone clicked yes.)
Yet at the same time, I'm happy when I can kick someone who's being a toxic A-hole, who's showing blatant disregard for other people's feelings or just being an overall hindrance to the group. But I agree with you - there needs to be better reasons. There really should be more options such as "Not playing to the game's mechanics' or just SOMETHING to say 'We feel we need a more skilled player in this slot."
And don't get me wrong - I've kicked people before.
I had one particularly toxic bard in Aurum Vale. He insisted I lost hate on the second boss (You know, the one with a hate reset?) and also blamed my tanking skill when he got backed into a corner and took a full 100 onze swing without even trying to dodge. He just stood there, and took it. This in and of itself wouldn't be bad, had he not proceeded to try and degrade me afterwards and started an argument in the party chat in regards to it. Ultimately, I voted to kick him, even though we downed the boss, just because he was bringing a generally vile atmosphere to the party at that point.
Perhaps when they start the 'abandon duty' option, they will add a 'Withdraw Request' option for party members who would like to be removed because they realise they are not up to par for the dungoun. An option where they can request to withdraw, and as long as 1-2 other people agree, they withdraw with no penalty (Which would be nice on a lot of regards.)
Anyways, hopefully your future runs go better~ <3 ^^