The thing is, they're not going to "believe" they're either sex until they reach the age where they develop and find out what they are. Before then, they might have particular hopes or about what they'll become, but they're not anything yet.
Personally, I think the fun part to play with is how they would regard this foreign-to-us approach to biology and gender perfectly normal, while our normal is bizarre to them.
At least as I headcanon it, my Viera is still rather unsettled at the prospect of other races' children having visible genders from birth – to him, it's as if they were born with adult features