Gladiators are all about swordplay, so you can only access their abilities when using a sword. Ditto with everything else, including crafting (you need a pick to mine, etc). As far as magic, its because you channel your magic through the weapon. Conjurors use wooden staves (they draw their magic from nature), for example, in comparison to Thaumaturges and their bone/metal staves. Arcanists read and channel magic from their books. Your class level is essentially how skilled you are at that tool/weapon. I heard (not sure if its true) in Japan, the class names are literally called like "Sword User" and "Lance User" instead of Gladiator (like Gladius) and Lancer.