I would say it may be a toss up between Dragoon and Monk for difficulty.
I have neither class but just from what i have read.
Bard is in fact the easiest and those claiming it is harder then blm, well you have the right to your opinion is all i will say