Why not just forget bonus exp and let us use Guardian's Aspect everytime?
Toogle it on and we get more exp and lose some GA points if we get exp (gathering, crafting and fighting)