Take a break and do some other things in game for couple of day after patch, the Caravan NPC would be the only one on your minimap after that.