Wow, 14 pages of useless arguments and a simple solution is obvious:
1. Open world PvP (optional in towns and other crowded places)
2. Only if both players agree to!
3. Ability to port to PvP arena at any time (for privacy or avoiding crowded places like towns etc.)
4. Ability to /ignorePvP - to avoid PvP request spamers.
5. Ability to /hideduels - makes duelists invisible to you, if there are bunch of them in a crowded place ruining your FPS.

Few simple things to make everyone happy. If both the duelists and the players around them have an option to avoid each other with a press of a button, what can they complain about?