I can't think of anything you're doing wrong, however there are certain challenges in the challenge log that don't give a chance of a recruit. Beast Tribe Quests, Retainer Ventures, Companions, Player Commendation, Wondrous Tails, and Overall Completion challenge log categories don't give a chance to recruit, so if most of the challenges you've completed fall in these categories then that could be a reason.

Otherwise it's probably just bad rng. Some of my FC mates had a similar hard time.