Being big doesn't necessarily make you better in battle, particularly when you take into consideration terrain. It provides brute strength yes and the ability to carry heavier weapons and armour but it comes at the cost of endurance and stability.

Further a large fighter often has greater issues with unstable or uneven ground. In highly muddy or unstable ground a heaver fighter will seriously struggle in keeping their feet making manoeuvring even harder. Further, enclosed spaces often cancel out a lot of the advantages size can give particularly if the larger fighter cant stand up straight. In narrow low ceiling tunnels a Roe would really struggle to fight effectively against a armed Lalafell.

However in truth the differences in gender would be genetic probably. No amount of training will change that. Unless the Au Ra specifically engaged in a breeding program to make bigger females you wouldn't see much difference in size regardless of condition. More likely Xaela culture would have adjusted roles in combat and such to match each genders size and strength. As I pointed out above, there are plenty of roles which smaller females would probably manage better than the males.