Happens to me as well, super weird and buggy feeling. No idea why things reset like that.

Of note, it happens to certain PvP UI elements after a time too. Always fun to load into Crystalline Conflict after a break from it and find my team and enemy bars have jumped around back to the default as the match start timer is ticking down.