The majority of damage is physical in most fights?

You lost me there. If most of the hard fights did mostly physical damage there wouldn't be such a huge demand for dragon kick from monk in raiding. The most devastating moves in this game are magic based.