I've noticed this as well, and CUL seems hated most by all Craft skills it would appear. The best advice I can provide when attempting to mastering 2-3 skills is to level all your available crafts to at least a skill of 5-10. By doing so, you spread the diminishing effect upon all your crafts randomly, and its impact isn't as cumulatively directed towards one's desired trade-skills.