Don't even look at stats until you're 50, then just go with higher ilvl until you can choose between the two i90 sets. People say 491 to get 6 skills in a rotation. I think that's silly. I think skill speed is a rather poor stat. I don't have any exact numbers of when it starts being bad, honestly it depends on the fight and party make up. Fights with little down time, a lot of aoe, and no bard Tp song make skill speed a terrible stat. You want close to 0. Fights with the opposite of those make it better, along the lines being on par with crit. The only magic number worth worrying about is accuracy. Get to the cap of whichever content you are on and then avoid gear that gives additional accuracy.

tl;dr - Plan around accuracy cap and don't get more than a few pieces of gear with skill speed.