It would need a real flexible system to solve that problem.

One idea:
If too many DPS are in the queue, party size would automatically increased to 5, with 3 DPS.
All mobs and bosses in the dungeon would have 30% more HP then.

this could basically work for up to 8 people, 1 tank, 1 healer and 6 DPS and the game adjusts the enemy HP accordingly.