For any class so far I've gotten the best bonus from choosing gear that gives accuracy, m. accuracy, attack, or m. potency over straight stats.

Second tier spells are definitely stronger than first tier, and you will notice a damage increase from INT and the elemental stats. It's just not as large as it can be. The difference of 100 points will make a difference. Adding 10 points will not.