That restriction is probably in place to prevent pre-made parties from fishing for the dungeon they want, negating the point of a Roulette being a Roulette.