It's not impossible for people to physically move from shard to shard, or from a shard to the Source. Cylva and Unukalhai (poor neglected Unukalhai...) are proof of that. It requires a lot of currently impossible to reproduce Ascian shenanigans to accomplish it though.