During the Elpis and Pandaemonium quests Azem is always referred to by the same gender pronouns as your character, but this is more clearer if you're playing a female character.
Additionally, unless you're playing as a lalafell, hrothgar or roegadyn (in other words, a more 'human' looking race), you're outright stated as being in "Azem's spitting image" (paraphrasing none other than
Emet) - of course viera and miq'ote's animal ears are convienently ignored in that regard.
So it's pretty clear that Azem's canon appearence is truly meant to resemble your character, but naturally behind the scenes during development this means the 'true' default appearence of them would be a male midlander hyur (to resemble 'Derplander'/Ardbert) however.
The process of reincarnation in the Lifestream ensures that when a soul is reborn their memories of their previous life are meant to be lost, but, some fragments can still linger, and we saw this with none other than Fandaniel - his last mortal form before his 'ascension' as the Ascian Fandaniel was the Allagan scientist Amon who, as the reincarnation of Hermes, inadvertently created a snake familiar with the same name as one Hermes did pre Sundering, without even knowing it (although he probably realized this belatedly after Emet raised him back to his office as an Ascian in the dying days of the Allag and thus restored his memories - arguably this is why Fandaniel ended up so unhinged).
After Fandaniel destroys himself and Zodiark his soul returns to the Lifestream, but it resets his very personality back to Amon (and he gleefully admits "of all the faces (he's), worn, this is his favourite!" just before fighting him as a spirit at the end of the Aitiascope), showing that a soul seems to accumulate memories with each reincarnation, and that it can affect the next incarnation in ways not known.
And on that particular point, is Azem's sense of altruism and wanderlust very much influenced the WoL's motives and drive even now, and even Emet basically tells the WoL to be more like his old friend and continue to explore the world, study other cultures and help people with their problems.... ironically pretty much what an adventurer does anyway.