Greetings and salutations! Happy to hear you're liking the game.
The duty finder is cross-server to your data centre. As you're on Masamune, your data centre is Mana, which iirc is a Japanese data centre.
If you have English selected in your Duty Finder options, you may not come across too many players.
For the record you can see what servers are in which centre here, http://na.finalfantasyxiv.com/lodestone/worldstatus/

I'm happy to invite you on over to Hyperion, on the Primal centre so you can play with little old King here, though we're sometimes closed off to new player creation due to being so active. It doesn't last all day (usually) but hey them's the breaks of being on an active server. Of course, I'd suggest any server on Primal or Aether, (Or Chaos if you're in the EU area) for optimal english play.

Then again you could just enable Japanese on your Duty Finder settings and play with your fellow Mana-mates, but if you don't read it, it may be troublesome.