You don't have to drop leather working. Just go pick up armorer at the guild in LL. You can level every class without having any negative effect on your other classes.