As said, this thing just pop'd in my head that's every skill cooldown (may be MP, TP as well) should be refreshed after a wipe.
If it is a restart, everything should be set to zero. I can't think of reason for the way it does currently.

Let alone dungeon because it may be tricky if we can reset cooldown for bigger mobs pull.