It's an annoying result of the way it's designed. When you only need one boss, it makes sense to try and get that boss rather than sit around somewhere that can't possibly get you anything.