Even if you don't want to do pvp, you can still do the quest that unlock it, and then forget about pvp afterwards.
Unlocking is optional, as is doing pvp even if you have unlocked it, but if you do the unlock quest you have one fewer unfinished quest around to annoy you.