When searching for a specific raid/idungeon, why does the duty finder sometimes seem to dump you into the 'general pool' of people using a random search after a certain duration, and other times, not?

I've been in a queue for Obronne Monastery for 65 minutes now. I am one hundred percent certain that other random raids (not OM) have launched in that time. But I appear to have been overlooked. I realize that looking for a specific raid or dungeon ups the wait times, but typically - after a certain duration - groups seem to be populated by people in the general, random queue. Not always, though - sometimes I just sit like this for 90+ minutes with no relief.

Why is the duty finder wired this way? Or, perhaps a better question is: what is the DF doing that sometimes it fills these raids really fast, and sometimes it just lets them sit? I'm not angry, I just want to know why the wait times vary so enormously.