The challenge argument is very different from the convenience argument. As people pointed out, finding your way around zones isn't hard (ironically the same people who say it should be a challenge). It's just inconvenient. It doesn't require some sophisticated tactics, good gear and skills to be pulled off, just a sense of direction and good communication. If someone describes the wrong place in Dynamis - Xarcabard, you don't find it, no matter how good your skills are. It would simply enhance the flow of the game.

If you think missing maps are challenging, why not remove maps from other zones too? Why not remove means of teleportation, when it's "harder" to walk? It's the same thing, it wouldn't be more challenging, just inconvenient.