The game checks things in certain intervals. Aggro on monsters can be a bit funky because of it.

Just try to be quicker on the adds, I guess.