Red Magi also use their Crystal Medium as an aetherial accelerator which allows them to get a lot more bang from smaller amounts of aether compared to White and Black magic.

The lore book does say that all the physical classes do use aether/magic as well even if said attack doesn't cost MP.