What lore do you ignore?

[Vyrerus]

That's not what Azem's crystal does. It is explicitly mentioned that it does not have any actual memories of Azem, only that one spell. It doesn't function like the others, because it wasn't designed in the same way or in the same batch.

It actually isn't explicitly stated anywhere with what you say. It is explicitly stated that it has Azem's spell, but it is also stated that it contains the name and role of the office. There's nothing that Hythlodaeus says that says it's different other than it was made in secret.

The issue lies in, it would be Emet-selch's memories of Azem just like how the other Convocation stones are said to be, "As remembered by the Unsundered."

Unless you're going off some additional context from another language. I haven't looked into the scene beyond English.