Is it just me or is this strange that the Guards before Merlwyb's Room or the other Leaders let you not visit their Rooms.
The Guards still think we're "Unknown Adventurers" even if you completed the Heavensward Story.

It's clear you have to earn their Trust and yet at some Point you should be able to visit their Areas after a Part of the Main Story.