As i have read through some things, i would say, why not try something like gathering stacks of different elements, depending on the en-spell used. Then depending on the count you have spells gain some special abilities.
And concerning different elemental usage there could be a elemental rotation like (earth >fire) (fire>air) (air>earth) or something. That was just thought out in a few minutes, so dont nail me on it, but the essence what i mean is
-put some elemental debuff on the target
-switch to the stronger element by using some kind of other skill that cycles through your elements, might as well be a weapon skill
-gather by using this elemental wheel rotation different elemental stacks
-spend them for better utility or nukes, or what ever.
Every job has a theme somewhere and maybe this inspires you to other thoughts.
But i like the idea of a spell weaving sword fighter, so thumbs up