character creation says that they hid in caves from the Hyur, and while most still hide in the caves a few come out to rob and pillage people. So that is probably why you don't see them anywhere.