They specifically said at the live letter "you can visit other worlds and do hunts there", so wanderers are doing nothing wrong.

Cross-World has been positive for hunts.

1. People who work now can now see S calls as frequently as A calls, and more realistically achieve their 2k.

2. Servers that struggled to kill their S due to a low population, now most definitely do not.

3. If you miss an S while in a dungeon or due to an early pull, get it from another server instead.

4. You may miss more hunts due to early pulls and fast kills, but there are more hunts to get overall, so it solves itself.

5. Meeting hunter counterparts from other servers has been a great experience, like the WoL meeting Noctis.

6. SE acknowledged they die too fast and indicated they were intending to make them FATEs mid-5.0 sometime.