SMN and, to a somewhat lesser extent, RDM are both jobs that are reliant on being able to weave skills between casts. SMN especially can require a lot of double-weaving, and so having an excess of SpSpd can be heavily detrimental as it can cause clipping in these cases. Both these jobs are relatively mobile and have relatively short cast times for spells, unless you make the mistake of hardcasting Veraero/Verthunder anytime aside from pre-pull. BLM, by contrast, relies far less on this kind of weaving and depends on SpSpd to compensate for the longer hardcasting times for its spells. Granted, I have no real play experience with BLM so I can't vouch too much on the specifics, but that's the gist of things as far as I'm aware.