One thing to keep in mind with AST is that your DPS (and your contribution in a group) is entirely balanced around you pretty much constantly using cards. This isn't going to make it a DPS but it will at least help it feel a little more like you're not doing trash tier DPS.

As for cast times in general, it really is about picking your spots and making use of the full kit of abilities you do have. With Black Mage in particular you do have extra tools to speed things up and your casts have a variety of cast times to manage, some faster than others so it's really about using the right tool in the right scenario. I do agree that it can be frustrating when you start a cast only to have your target die but some of that can be mitigated by being more conscious in your target selection. Focus on the highest HP target first and rotate through the pack to select your next target based on estimation of kill time and your cast time. In some cases it's better to just use a shorter cast, weaker spell instead of a big boom if you know there's a good chance your big boom won't go off anyway.

If you want a more pacey caster, you could try Red Mage. Because of their dual cast trait you get a lot more of a steady rhythmic caster, plus your melee combo can break you away from the potential tedium of doing nothing but watching cast bars if that's an issue for you.

In the end, I find myself enjoying casters because of the cast timers. It requires more thought and planning to make the most out what they can do, and the punishment for "doing it wrong" is very clear.