Your theory sounds like a theory of the Dwarves disappearance in The Elder Scrolls universe...

Though wasn't something about what happened explained during the CT2 storyline?