We do it all the time, *roams into a camp* " Oh ok guys there's already a party here let's go somewhere else" No its not in the rules but, When you're in a restaurant waiting for a seat, when somone gets up....do you just go and sit down? One would hope not..Common courtesy