Not at all.
I mean, why should we look alike? If we were going to share an appearance, we should probably look like Azem, as s/he is the original "us". But, as I understand it, Azem's soul was shattered into pieces, of which we are one. There's really no reason that Ardbert's and our souls would have to be identical (similar, sure, but not identical) when we're fragments of the same whole, not carbon copies of it. If it makes sense that even our formless, genderless souls aren't exact copies, then there's no reason at all for the fleshy wrappings around those souls to be. Especially considering the fact that we've probably been reincarnated a million times anyway between the original Azem and our current form.