I think that question was answered. It's not always going to be from a different server, just most of the time. Duty Finder assembles its groups from all players in your data center. So the odds are that you're going to end up with a group of people from multiple servers.
I think Bixby is spot on here, this may just be a case of confirmation bias. You're remembering when things went south and in those runs the players were from other servers. Sure you may have had a few bad ones in a row. It happens to all of us.
As of right now, Square doesn't have a means for us to blacklist players from other servers, just our own. And even then, as Sadana points out, you still can be grouped with even those people in Duty finder, so the blacklist doesn't help.