I think that's done to stop you from using it as a 20% damage reduction for a single big attack and then immediately swapping back for the smaller attacks. Defiance doesn't cost mana, but it does have a decently long cooldown, and you cannot turn it off while that timer is cycling.