The 'lore' was always a flimsy excuse in my opinion. It's constantly changing and based on the premise surrounding 6.0 and the global threat it represents...that'd be a pretty good excuse to justify adding the missing genders as a playable option. All it really needs is a line of dialogue suggesting that a handful of male viera/female hrothgar quietly left their respective homelands in order to become adventurers. That covers the ARR to ShB time period. For EW, the reasoning can be that many more male viera/female hrothgar were driven out due to their homes being set on fire and attacked by the new threat.