The retainers seem to be time-based and not entirely random, as I've seen one by the node for Fragrant Logs in South Shroud several times while gathering said item.