You have to have done the level 10 quest for your class, that's the only requirement. They actually say as you're handing that quest in that you can now take different classes.

To summarize:

-Your main quest progression doesn't matter
-It's the level 10 class quest for your starting class that matters
-You must complete that quest (and by extension obviously the level 5 quest as well) to unlock the Armory system.