Your level largely doesn't matter as far as being overleveled goes. Stats come from gear and the majority of important content will level sync you.

If you REALLY don't want the buff, you need to make your character on a non-preferred server