Not sure if it works the same way as DF, but aren't you only penalized after the 3rd time of missing the queue pop?