I've started the quest 'A Secret From Everyone'. It wants me to go to the 'house of letters' and I can see it on my map but I can't get into it. The door doesn't open, I tried that little path at around x34 y26 but it don't let me go over the boulders, and I've tried to go the long way around the cliffs and it still won't let me in.

Any help? It's getting frustrating.