What you think are magical attacks are actually physical, the game is weird like that at times but Parry only works on Physical attacks. I know what you mean but trust me when I say you haven't parried a single magical attack.
The problem is that it's RNG based and extremely unreliable. The scaling is so bad it's best to rely on skills that boost Parry chance rather than the parry stat itself. Parry is negated by Critical hits and you have to wrap your head around the idea that one of the best ways to mitigate damage in this game is to kill whatever is hurting you faster. Giving a boost to a stat that gives you a more sizeable boost to damage is more useful than a stat that MIGHT give you a small boost to your defense. On top of that most big fights are scripted so big reliable mitigitations are far more important than RNG mitigation.