If I remember correctly, during the Black Mage questline, they state that casting without a proper focus can cause great harm to the caster as they are channelling the focused aether through their body instead of an object. Might have even been the original Thaumaturge quests. Not 100% on this, but i definitely remember the focus being VERY important.