Skill Speed also effects the amount of damage that auto attacks deal.

Spell Speed, I've been saying for a while, should have been set to increase MP regeneration in addition to cast time, but now that piety does that, I don't think we'll be seeing that adjustment to the skill.