So I've been on at least thirteen separate servers, with at least two more than once. Let me break them down.

Exodus: This was where I first started. Wasn't too bad, but I think it's deadened over there since it's been on the preferred server list several times now.
Brynhildr: Wasn't too bad, but left because there seemed like there was going to be a civil war in the Lalafell community.
Lich: When I was there, it was pretty much as dead as the thing it was named after.
Faerie: Server was okay. Was more of a case of being there for people who had later decided to quit playing.
Balmung: As much fun as I make of it, it's surprisingly not that bad of a server so long as you're not expecting things like housing, and I even hope to come back there one day. Just um....avoid Ul'dah at all costs if you can.
Cactuar: It's sorta like Portland, Oregon in that it just produces its own brand of weird. Not bad, just weird.
Moogle: Very much a French server nowadays. Lalafell community there is a bit too much into loli stuff.
Omega: I went there when it first opened up, and I was probably the happiest I've ever been in the game the day after that 90-day transfer lockout had lifted.
Louisoix: Wasn't too bad when I was there, but no real incentive for me to go back there.
Lamia: Not a bad server, but there was one guy in Limsa who was such a public nuisance that someone started a GoFundMe to pay for that guy to be transferred off the server.
Midgardsormr: Another server that isn't too bad, but not really one I'd feel like going back to.
Mateus: I'd honestly like it if wasn't for the fact that everyone seems to expect everyone to be an RP'er. If you like to RP, then it's the server for you. Otherwise, don't bother with it.
Adamantoise: This is where I am right now, and it's nice. Will I transfer off it when we get the free transfer period? Yup.

TL;DR: Omega has probably been hands-down the single worst server I've ever been on, but it's been a while since I was there and I'm not sure if there's been any change in the population there.