Did you die in the duty? If so, the minion still despawns.