If there were specific dueling grounds where it was enabled, and thus anyone who wants to opt out can just choose not to go there? Sure, why not. Just like Triple Triad being restricted to certain locations.

If it were anywhere someone felt like throwing down a challenge? Not interested. It got old in WoW to have high-level people pushing low-levels into it just for shits and giggles, and it'd get old here.