Skillspeed has pretty much always been a case of "get as much as you need for your rotation to work in a fight".
The only time where going all in on skillspeed was viable on some jobs was back in Stormblood and only because we had access to the completely busted Elemental+1 gear.