"Join in Progress Only" has issues aplenty. If the majority of players are selecting that, more and more instances aren't going to be started. Wait times increase for everyone involved.

Placing a minute or two restriction if you withdraw from a duty queue would be a good way to start addressing the problem.