As far as I know it's not like Final Fantasy XI where you were given a random server assignment. The only thing that will keep them from joining your server is if it's full before they can join, as far as I know.

You should be able to choose the server in which you wish to play.