Are Duty Finder queues influenced by whether you have finished the duty or not?

Yesterday I unlocked Ifrit HM and did it for the first times.

First try: <10 minute queue: Sadly a fail.
Second try: <10 minute queue: Succeeded, progressed story quest.
Third try: >20 minute queue: Cancelled queue because I would need to go 1h30 later.
Fourth try: >20 minute queue: And still counting right now

Does it matter that I completed it once? Does the duty finder put me in front of the queue if you need it for story quest?

If your DPS then those queues sound pretty good time wise. I don't believe the DF does anything other than first come first served. I've never experienced longer delays for content I've already cleared before.