So what exactly is driving the developers to keep going?

[Fawkes]

It made me curious about something else as well, let say during Stormblood that Endwalker was actually being made aside from Shadowbringers at the time. Does his team actually plan a lot of things out way ahead of time and thus why they can meet Patch Deadlines so easily like that?

Because if so they know how to keep secrets.

I'm pretty sure the plan for after Stormblood was always to go straight to Garlemald next, but then Yoshi-P decided on a whim to make Shadowbringers instead.

Also I think in some pre-expansion talks they mentioned what percent of the new maps were done at that time. I'm pretty sure they're always just working on the next thing and not anything further down the line.