The Forum in Sharlayan knows because they know what the Loporrits know. The Eorzea Encyclopedia also mentions the previous ages and gives them each a brief description. It is written from an in-universe point-of-view so I would assume some scholars know as well.