I am curious, when i upgraded my animus stardust rod, i had 30+ Det and 40+ Spell Speed

Went with 33 Acc 22 Spell Speed 11 det and 9 crit

When i upgraded my spell speed and determination both decreased to the level i put into the infusion.

Could i have saved those stats? did i goof my relic?