Swap to a crafter set and just start making a recipe. That should scare them off quickly enough!