Remember that multiple Umbral Eras happened between the Allagan Empire and when Nym/Amdapor rose. Also remember that the Crystal Tower was buried underground until the Calamity. It's pretty safe to say that the magic the Allagans discovered were lost, and have only recently been rediscovered. Other types of magic were independently discovered by these various groups.

Always remember that the history presented in this game is a bit fragmented due to multiple cataclysmic events that have happened. A thing that's stated by people in the game may not be necessarily true when we as players have more info than them.