Might be you didn't conclude the quest, check your journal.

It might be you need to go to the Crystal Tower Entrance (Porter), to interact with an NPC (maybe?). I don't remember the details, I hope you resolved the problem.