I thought about it for a while, and I came up with this:

Instead of having maps pre-made for you, you uncover the area yourself and draw out your own map in your head. You remember the terrain for a while, but it turns unknown again after a while until you visit the area again. There could be a quest where to earn map drawing materials, so instead of having to remember then, you draw them out as you travel and then can always see it.