I feel as though we are overlooking a commonly used object throughout the plot: auracite.
To Cleretic's point, the key in all likelihood was not made by the Ancients, just as the Heart of Sabik and the Zodiac Stones weren't. It's true that if the key was used for interdimensional travel, it would've served little purpose in the unsundered world, which is why I believe that isn't its function. Interdimensional fusion and the ability to travel between shards is just something within the key's capacity to do in the sundered shards. Auracite is said to "make manifest one's most fervent desires" and given how the Milalla used it to escape a calamity, and Sphene used it to fuse a part of their shard to the Source as well as transform, its uses can vary. It is not solely a tool used for traveling, by the Traveler. In the exact same way, the Heart of Sabik never had one use-case nor has any other piece of auracite in the game, such as: white auracite to contain and destroy Ascians; the same white auracite to transfer souls to and from the First; the Zodiac Stones; the warmachina of Werlyt; even job stones.
Moreover, given how powerful we know it to be now, it could very well be that it has something to do with whatever Azem was doing during the Final Days. I have to believe that if Azem truly did defect as is claimed, what could they have felt was more important than Hydaelyn or Zodiark? What threat did Azem see looming that would overshadow everything the Ancients were capable of?

Originally Posted by
Cleretic
That implies that someone post-Sundering knew who Azem was enough to invoke their title and history--which itself would be interesting, because everything we've been told suggests that the knowledge of Amaurot was lost to vague dreams near-immediately.
Have we considered that the Milalla, being in possession of the key, were perhaps friends to Azem in some capacity, or venerated Azem not unlike the Xaela?
We only know they disappeared some 5,500-6,000 years ago, right? That says nothing of how long they had been around. My other thought is that maybe they were the ones who stumbled upon the ruins of Amaurot in the Source. They may have found Azem's key and learned of the Ancients there. They share some commonalities with the Ancients, such as their masks in the chamber and the title of Speaker, enough that I can't immediately dismiss they may have been exposed to each other in some way.