I am not saying it is okay to ruin the game experience of others.

If you want to make PvP worth for just the sake of PvP, don't add Esoterics or Lore or any other PvE rewards to it. As simple as it is.