Let them kick you if they're going to be dicks, they can have fun sitting around waiting for the in progress queue to pop, and you can instantly get into a fresh run with some less impatient people.

Also, start doing the Crystal Tower raids as soon as possible, they're very forgiving even for people who barely meet the item level requirement, and are a great way to get geared fast.