After the sundering no one had any memories. Did they just start out as cavemen having to rediscover fire and all that?

The First seemed to have the same level of technology as most of Eorzea. But time worked differently there except when it didn't. And they were a post apocalyptic fantasy setting so it is possible they lost a lot of tech.

If Hydaelyn could specifically decide what people remembered and what they forgot, then her power is way way way way stronger and precise than we give her credit for. At which point why even sunder people? Why not just reprogram their memories with what you want?

This is why I think that post sundering, everyone was for all intents and purposes, a caveman. Hydaelyn then could have taught them via visions and what not the level of tech she believed they were worthy of having and cement her position as a goddess and explain why everyone knows about her and worships her.