As far as I know this server delay exists to help people who have a bad connection or are just having some lag issues so their performance in PvE will be equal compared to those with really good internet connections.
This system does seem to help a lot in PvE and doesn't bother me at all. In PvP however it has a reverse effect and it's more frustrating than it does any good.

I don't think there's much that SE can do about this.
Would be nice if they could remove this server delay for PvP only.