Gender locked race?

[Senn]

Also I think it might be to better explain why the males are allowed in the village until puberty, at which point their change is enough that their society has been built around the idea of males leaving the city for good reason.

Or it could be that the males remain in the village up until their society notices they aren't growing any bigger as opposed to the females.