Difficult is subjective to the player.

Personally;

SMN = RDM = SAM = DRG = RPR
MNK = MCH = BRD = DNC = NIN = BLM

I'm just bad at remembering my proc, that's why all ranged physical are a level higher. Anything that is a script is pretty braindead and that sums up SMN/RDM/SAM/DRG/RPR for me.

In term of jobs having core issue in their kits which affects the difficulty.

SMN simply cannot optimize. It is impossible. You have to resort with cursed knowledge to fit those 6 GCDs on your demi-phases depending on your ping.
MNK requires God ping and you have to resort to cursed knowledge.
MCH induces carpal tunnel for no gain.
BLM makes people adjust for them so it isn't that difficult. The true difficult and villain of BLM is when a group refuses to let the BLM optimize which is why I don't play/prog with BLM.

There isn't truer truth that prog casters are just braindead and RDM just get it awful because of fight design. RDM is actually pretty easy and smooth to play in DSR.