Servers are grouped together in bunches called datacenters. There are currently 6 of them: Aether, Chaos, Elemental, Gaia, Mana and Primal. When you queue solo or in an incomplete party in Duty Finder you are paired with other players in your datacenter (not server) queuing for the same duty. As you are on Phoenix, which is part of the Chaos Datacenter, you will be paired with players from Cerberus, Lich, Moogle, Odin, Ragnarok, Shiva, and Zodiark in addition to Phoenix players.

As for why they were rude? Different servers have different cultures and player attitudes. Yesterday I was paired with a Blackmage from Balmung(one of the unoffical RP servers) for T7. He had macroed messages to every single one of his actions making him spam party chat with what amounted to announcements that he was casting Fire.