Currently yes, the only 2-Hander melee DPS is DRG.

I mainly play melee DPS in MMOs, my main in WoW was an Enhancement Shaman, but one of my alts was a Mage. My Main in this game is DRG, but I've also leveled a bunch of other jobs, including BLM. Not quite sure what the delay is you're feeling. Which caster did you try? Perhaps it's just the animations? Honestly the combat anims in WoW don't have many frames, so they feel very quick.

Being able to move before the cast finishes is actually beneficial to learn as a caster, especially as a BLM. You'll usually see people refer to it as "Slip Casting" or "Slide Casting". It's useful for dodging and movement heavy mechanics.

It you are enjoying melee, I say stick with it, but if a caster is something that interests you it can definitely be a nice change of pace to switch between the two when you feel like it. SMN would probably "feel" faster than BLM because they deal mainly with DoTs instead of larger nukes. Just a thought.