The limitations come from a concept of specialization. When you become a warrior or a paladin you no longer practice the same skills you did when you were a generic arena fighter. You have to spend more time learning and mastering the new skills that are particular to your new field. That is why we do not have access to everything anymore. Think of the old concept of "Jack of all trades, master of none." You basically want a master of all trades idea here and while I understand you want to limit the skills to only those in your cross classes it still it just too much. The entire idea of specializing goes completely out the window.

I realize we all want to be all powerful super uber god characters but truth be told, being truly a hero and worthy of any kind of recognition comes more from using what you have and figuring out ways to conquer even with limitations. That is why I never liked Superman, yay... immortal super hero... how heroic is it to run into a burning building when you are impervious to anything? Work within your limitations, master them, overcome them. That is the idea of challenge, not being able to do everything.