Well, my current theory, based on the latest trailer, is that Eorzea (and its Shard equivalents) was the location of the first summoning, and subsequent Sundering when Hydaelyn and Zodiark fell out. It's the epicenter of everything, it's retained some mythological importance across all Shards as a result of that. I believe even the Ascians hold Eorzea in some higher regard than the rest of the world? So yes, there must be some other reason for that.
Now, that perhaps debunks my own Allagan/Crystal Tower theory, but... How do people on the Source remember anything? So far there's no reason to believe the First has experienced Calamities, it should have had a fairly typical history. Naturally the memory from the Sundering should carry on in some form (there are even cave paintings detailing it) so they have plenty with which to paint mythology with. The Source though, it gets hard resets on its civilizations with every Rejoining. How does Eorzea remember that Silvertear holds any significance with that happening? Simple, it's remembered as an important site, not because the details are remembered, but because civilizations keep setting up there. Why did the Allagan set up shop there? Because of a vague memory from a previous era that the area held significance. Why did the Mhach set up nearby? Given their ties to Voidsent, I could see that being a large factor given the Allagan set up a massive Voidgate to open there. Now, we remember it as being important because it was important to those civilizations that we barely remember, but do we remember why they found it important? Not really, so we make that up, and in a world of magic and aether? Naturally those are the go-tos for myth.