My theory is that it removes the item, then wears the other item, and doesn't necessarily swap the item, which is why it's such a pain. After you swap, are your counts the same? Otherwise, you may have some builds not using up to date armor sets.