well...

as I calmed down and talked to my party member, the party leader admits commencing a DF without me knowing and then when it quickly pops, the tank in party quickly cancelled since he want to take a break, an action that resulted every party member got penalize. It was all happened when I'm still in loading screen from last dungeon.

At least you know what happened. Conclusion: tell all your party member not to hastily register/cancel without other concern.