You cannot change equipped items or class of a retainer unless you cancel the venture. I made mine alt classes like fisher to gather materials I can't myself.