I like the current system, it just needs some limitations, like only being able to draw from 2-3 different classes on top of your main. Mix that with a few other changes, like making more spells worthwhile and affinity balancing, and you've got yourself a solid way to do things.