Just remove achievements and mounts or make them buyable otherwise. PvP will be gone then anyway (it even now almost is, you can't enter arena and frontlines take forever). How many are playing them for fun?

PvP is full of bots, win traders, leechers, roulette players (since 100/40 a day in a few minutes is nice to have), who don't care and even cheaters (no, I won't link to hacks, but of course EVERY MMO got them). This just means: it's no PvP at all...