I have accidentally pressed the add member button trying to exit the dungeon before.

I feel bad for the poor sob, but it has been due to bad timing where the popup is displayed a split-second before trying to "use" the exit stream, while playing with a controller.