I am pretty sure that this is a bug that has been reported before. When someone withdraws from a party after it pops, sometimes you don't get properly re-queued. Sometimes the system will pick you back up, but often it simply doesn't. As a rule, if your queue pops, it should keep popping with some frequency. If your queue stops popping for more than about the average wait time, you should definitely withdraw and re-queue.