I could be misremembering because it's been a long time, but I believe it was explained that people with the echo can teleport and everyone else can't.

So, everyone else is taking the long way, every time. Which is why they "journey with us" they are always asking to meet us at certain locations, because the WOL travels differently and it conveniently fits the quest structure programed to the game.