I didn't register that it said "physical" damage. That is why I have been seeing damage done anyway, from AOEs and such.