Can't speak for the caster classes, but I've played the melee classes a bit. Between DRG and BRD, I'd say DRG is more difficult, but no class so far has been overly difficult to master. BRD has a proc-based priority system, but no chains; DRG has a few chains, 2 of which are positional (one flank, the other rear) and one that's any position - so it's not too difficult to stay in an enemy's rear or flank and then just strafe a bit to get all of your chains off.

Not sure why there are so many bad DRGs, but also not sure why BRD has the "faceroll" stigma either. A well-played BRD maximizes their dps by taking advantage of Bloodletter procs in the GCD timeframe, while keeping their 2 DoTs on the enemy, and then fishing for a heavy shot procs.

TLDR: no class is a 10/10 difficulty, but none are a 1/10 either. IMO, the melee difficulty is MNK > DRG > BRD, but that's purely subjective.