Makes sense for the men to be the stronger of the two genders, honestly. Why else would you make the rarer gender - that is obviously needed for reproduction - be the one that is having an extreme lifestyle where most die during their own training? It makes no sense. Considering the lore has it that only the strongest survive and are allowed to breed then you're talking about the 'creme of the crop' type of deal.

So, yeah... it paints the image that the men are the stronger and more ferocious of the two genders - but the females are fully capable of handling themselves as well. But most of them simply don't wish to and prefer to protect the children and take care of the forest. While the men, that are extreme survivalists are trained to kill anything that sets foot in their woodlands that isn't a Viera - or has a Viera chaperone.

They could completely change this around however and just be all, 'nah, men are out in the woods just because' - but that doesn't make for good storytelling.

So, I personally think the males will have a more muscular build than the females. In regards to the height, this is an unknown as they can fit anywhere. They could be like Bosmer or the other extreme of wood elves.