Ideally, for the future, I'd really like to be able to manually connect with players on other servers instead of having the duty finder give me random people, in order to play some with my Japanese friends.
This isn't going to happen. Create a character on those other servers if you want to play with specific people. The duty finder is a matchmaking system essentially