I leveled all my crafters and gatherer with just two retainers. You need to manage it better.

Having a lot of space won't fix the problem for two reasons.
1. Having more space will just make you go "This is still not enough"
2. Market supply to be inflated, so when it comes to the material availability it'll just making things too cheap if a lot of them is available.