It obviously does something since we're the only thing stopping the weekly world ending threat and it makes the other characters feel weak when they inevitably have to run to character to accomplish anything. Toning down our characters power would allow for greater variety in story too since every threat wouldn't have to have some excuse to be so powerful that only we could defeat it.
Why would any enemy that knows who we are willingly fight us at this point? Even the latest story quest has to have our character dump aether into Alisae so there is an excuse as to why you don't effortlessly slaughter everything there, and it's getting a bit boring.