I strongly agree here. I was never a fan of PvP, if anything, playing on a PvP server in WoW is what made me hate that environment, but XIV's is rather... interesting. It got me by surprise that the way classes play is set up differently that they would in PvE, and in a way it makes it more lightweight to rely more on movement and quick action instead of long pre-calculated rotations. I'm as casual as it gets so I won't even bother with rankings or job balance, but as far as I can tell, XIV PvP is decent.
That said, PvP as a genre is just not my cup of tea and would normally avoid it unless necessary.