You have to run away from them for a little until they de-aggro.