If you use the Duty Finder for instanced content like dungeons or guildhests, it can put you together with players from any of the servers in the same Data Center. So if that's your main reason for wanting a more populated server, it doesn't really apply. From any of those, you'd have access to being grouped with people from any of the Chaos Data Center's worlds (Cerberus / Lich / Moogle / Odin / Phoenix / Ragnarok / Shiva).

It's different if you want to put together your own team of people you know. If you're setting up a group yourself and selecting members to be in it, then you can only accept people from your own server. That's where the specific server you're on makes a difference (and it's why people with friends in the game want to be on the same server as their friends). But for that, it isn't about the overall population of the server as much as the number of players in your Free Company and/or Linkshells (i.e. the number of people you know or can easily reach to ask them to join you). Even on a relatively low populated server, it should be possible to find an active FC to join.