As Elidibus' servant, I'm... pretty sure he was just hopping around various other worlds to maintain the balance as well. Maybe. Sounds like the most reasonable explanation.

That raises other questions, like why he was sent to the Source instead of the First Shard (which was, and plausibly still is, in danger of becoming a Void), but that's the best I got.