According to the Lodestone (Which of course isn't 100% accurate by any means), the vast majority of the NA servers have 25k+ people on them. That's hardly an issue with population, and it would cause a ton of problems due to the increased stress. Maybe if very few people are doing endgame, it's a problem with endgame.

EDIT: http://xivsoul.com/