If you dont like the people are end up in a party with you have many options

1. Leave - annoying but u can craft etc.
2. Blacklist or Block so you are not in a party with them again
3. In rare circumstances and where warranted you can report them

People say x and y about complaints but realistically its up to you to ensure you can be in the best situation not to let it happen again.