The skill speed stat reduces the cooldown, so it will be lower as you get better gear.