The following are my experiences from being on both a small and large server, and anecdotal information from these forums over the years.

It depends on what you are looking for:
You want ERP - head over to Mateus (Balmung is still locked)
You want a small community - head on over to the preferred worlds Perferred / Congested worlds
You want a large server - come on over to Gilgamesh or head to Sargatanas
You want a better economy go to a midsize server like Adamantoise or Midgardsormr - midsize have better economies than large because the prices are higher and there is enough volume to turn a good profit. Large server generally have lower prices (more competition) and faster sales. Small server have so many new players, that a disproportionately small number of players have spare Gil.

With the cross world PF, and LS; large server raid communitiy has been rendered irrelevant.