I generally prefer to do all the sidequests as I unlock them, primarily because they tend to make the most sense taking place around the time they're unlocked (all sidequests are unlocked by MSQ progress, save for a few in each town). As others have said, there are a TON of options to get xp for other classes, most of which benefit from the +100% xp bonus from your armory bonus, which sidequests do NOT benefit from.