The recovery of your TP isn't tied to Skill Speed, so more Skill Speed means you'd be using skills faster and without more recovery on each tick.
You'd have learned Invigorate around level 22 on your Lancer (which by the way you should get your Dragoon soul for!), which helps recover TP without needing to leave battle. Use Invigorate when you reach a bit less than half your TP as for Lancer/Dragoon specifically it'll restore 500 points.

Otherwise in longer fights where TP is an endurance race, you'll want a supporting class, a Bard or Machinist, to recover your TP for you. Ninja also has an ability for this as well. But at your level you wouldn't have much need for them just yet.