It's not really a "UX miss that could have been caught with testing" because it's a deliberate decision targeting a different cohort, choosing to preserve bar layouts rather than overwriting them when a revamp changes things. It's the same reason that invalid buttons stay on your bar after an expansion changes them, so you remember what was placed where and can adapt to the change easily. 100% the decision was discussed and they decided that the small amount of people who would a) blindly queue in without checking and b) couldn't figure out where their PVP profile was to assign new buttons when they realized their buttons didn't work wasn't worth sacrificing the existing PVP bar layouts of the entire playerbase for.

A tutorial would have been a nice way to avoid it, but sacrifices get made sometimes.