Its about the Duskwight Elezen.

[ObsidianFire]

As far as I know, we havn't killed any elementals. In fact, the entire Conjurer and White Mage storylines revolve around making sure the Elementals aren't tainted. Because when Elementals go bad, they go really bad (potentially causing an Umbral Calamity bad). So Grindania is perpetually in an "an ounce of prevention is worth a pound of cure" scenario. The reason Gridinia has to take the stance it does towards poachers, illegal logging (Ixal), etc. is because if they don't, they run the risk of the Elementals throwing everyone out of the Twelveswood. And that's if they're "nice" enough to not kill everyone. The reasons why the Gridanians don't want to let the Duskwight and Keepers do certain things isn't necessarily because they don't like the Duskwight and Keepers. It's because letting certain things go unchecked could lead to the Forest Elementals turning on everyone and killing them.

This puts Gridania in a bit of a weird spot when compared with Limsa and Ul'dah. All three city-states are portrayed as having problems with them. However, Limsa's and Ul'dah's problems are very human in origin. This leads to the potential fixes to those problems being human as well. Gridania's problems ultimately have very little to with what people want and everything to do with what a bunch of inhuman energy beings want. And the inhuman energy beings have all the power in the relationship. There's very little mortal humans can do to try to fix Gridania's problems that doesn't amount to "do what the Elementals say".

Edit about Gelmora: From both the Lore Book and various lore interviews, it seems that Gelmora used to be the size of Gridania at least. Having a good half to two-thirds of the entire population leave at once (all the humans and half the elezen let's say) would decimate it. It would simply be physically impossible for 1/3 of the the remaining population to do all the work the entire population once did. Stuff doesn't get repaired, roofs cave in, enough resources can't be gathered, etc. Gelmora would probably have various parts of it cut off as tunnels collapsed and overtime it would get worse and worse as facilities broke down. It would probably also have to deal with people constantly leaving as conditions worsened and word of how the above-ground was now safe to live on trickled down. Eventually the population would be too small to sustain itself and the city would have died out.

There's plenty of real-world versions of this to draw from. Several of which have no known reason for the population to leave the city they were living in behind (Montezuma Castle would be a good example).