Leveling your first job to 50 will always be slower than leveling alts because of the armory bonus exp system. If you have a DoW/DoM class leveled to 50, then all other DoW/DoM classes you level will get a permanent 50% bonus exp when killing mobs. It makes a significant difference when fate grinding.

I don't know how effective Arcanists/Whitemages are in fates but as long as you join a good party it shouldn't be too much of an issue.