There might be a certain number for skill speed you might want to reach(on bard and dragoon it's ~600) - and similar for accuracy. After that though crit and det all the way really.