A far fetched what-if theory.

[Vivi_Bushido]

This theory is split into two what-if ideas.

What if the First is filled with living Allagans who survived. Just like how the Garleans is an Ascian nation, the Allagans could be a Sin Eater nation. Crystal Tower is more than just a beacon in the First. There is clearly a city at it's base and an Allagan city makes more sense to me than an expansion to the Rising Stone. It definitely seems like we're going to get more Allagan lore with the inclusion of Crystal Tower's involvement.

The second what-if theory is, what if the white robed Ascians are sin eaters from the First? And Hydalyn is seen as evil there like how Zodiark is seen as evil here; and we find out that Hydalyn is just as bad as Zodiark. Then we realize that the Warrior of Light, a person who is always righteous. Hydalyn's chosen champion is actually a Sin Eater themselves this whole time. Which is why we have no past at the start of the game. We're like a white robed ascian just without the robe. The CS at the very start the game even shows you just poofing into the realm in the Crystal world. And instantly attacked by a dark Ascian. What if this was when we were really born and we never had a childhood. Never had parents. We just poofed into existence. We're just a sin eater for Hydalyn.

[Vivi_Bushido]

Also some neglected foreshadowing I just found.

[Yuyuka3]

I think you are overthinking this.
First, I doubt the Allagans still exist in the first; it's more likely they never got there. CT's original purpose seems to be opening a rift to the Void, not enable shard hopping in general. Also, CT was at the epicenter of the great quake, so that would make it a very dangerous place to be...
Besides, we got some info on the city at the foot of the CT in the First. It was found x numbers of years after the CT appeared in the First, it's a melting pot of different races, and it's purpose is to provide shelter from the sin eaters. Nothing really screams 'Allagan city' to me.

Second, no. Definitely not. The Warriors of Darkness are from the First; and before they became the Warriors of Darkness, came to the Source and tried to kill us, they where the Warriors of Light, just like us. Therefore I would wager Hydaelyn is either unknown to the First, as she was for a very long time on the Source, or she is referred as the Mothercrystal there, too.
Besides, the "no past" thing is explained as the Devs wanting us to have the chance to flesh out our own character's background, instead of having it provided to us. Besides, we don't suddenly appear in the cart or on the shop, or the trader would have been surprised about our sudden appearance. It looks more like we got on the cart at one point, likely paid for the passage, then fell asleep and what you call 'pop into existence' was a dream we had, or at least we initially think it was.

[Cilia]

As I understand it...

The "city" around the Crystal Tower on the First (Norvrandt) is populated by refugees from the Flood of Light and Sin Eaters. It only popped up (relatively) recently and doesn't appear to have anything to do with Allag.

Second, Hydaelyn is known in Norvrandt - "Arbert" specifically calls Her out for not answering their prayers for deliverance when we finally get another face-to-face with Minfilia / the Word of the Mother.