Quote Originally Posted by MiniTofu View Post
snip
The reason why is explained later, but the gist of it is that it's necessary for you to be here unless you want to lose to Garlemald.

The others had their souls sucked out of their bodies and need to be rejoined properly to leave, while the Warrior of Light was grabbed body and all. Therefore, you can travel between the two worlds with greater ease.