Don't forget that Duty Finder works by picking players from a pool of servers in the same data center. According to
http://xivsoul.com/, Tonberry is also grouped with Aegis, Atomos, Carbuncle, Garuda, Gungnir, Kujata, Typhon, and Unicorn - all of which are majorly Japanese populated servers. If you queue in E only, chances are most if not all will be from Tonberry (I once got grouped into an EN T5 group on DF, with a few players from Aegis). Whereas if you queue for J, there's a larger pool of players to choose from, and this is what it makes it feel like more Japanese players.