Au Ra physiology and other lore.

[Nataruma]

The Buduga seem like enough of a terrifying force to either mean the Borlaaq have something that makes them able to match foreign threats physically or something.

That something may be stronger magic. This is my own headcanon here, but I'm thinking the women may be stronger candidates for magical pursuits, this isn't to say that the males aren't suited or can't, but women may be better at it in most cases, at least that's the feeling I'm getting from their appearance and their personalities (what little there are to read about).