The reason why you stay queued for 30 minutes with people dropping is because NO ONE is queuing for NEW content. EVERYONE is queuing for In-Progress, and they ditch when they get put in a new one. So, if you removed the fishers, trust me you will STILL sit for 30 minutes queued, but with the difference than now instead of getting a pop/withdraw, you won't get a single pop and your queue will take forever.
If people were queuing for NEW instances, you would NOT be waiting 30 minutes, would you now?