Have you tried with different weapons to see if they scale off your magic damage from weapons?