As I understand it, when you queue for an instance, the "forming party" window will only track other players directly queuing for the same instance. But the queue is itself queued up to wait its turn to be filled by the roulette. So, unless you miraculously get a full party all directly queuing for that instance, you're just stuck waiting until your queue is the oldest one and next in line to be filled from roulette participants.

At least is how I suppose it works.