That's plain rude of him. Don't worry too much. You'll run into assholes wherever you go, just remember their name and avoid them so you can continue enjoying your adventures without having to frown or have your mood ruined.

If people who sent tell or asked to join doesn't meet my requirement or preference, I always make sure to tell them I'm sorry but I can't let you join. At least it lets the person know they won't get in, and so they can start looking for other group instead of wasting their time waiting for my reply forever.