Even if you select all as an English speaker it won't group you with those that haven't selected the English option. If you select more than just one language that doesn't mean you know them, it means you're willing to group with them and deal with any potential communication issue.

Anyone that complains that you aren't fluent in their language is an hypocrite.

Besides, some of the most fun I had back when I played FFXI was being the only English speaker in a Japanese group working with auto-translator to communicate.