We don't see all of the Rak'tika Greatwood, and in a sideqest the female Viera say that the males live hidden in a village deeper in the woods.

That's your explanation.