That is definitely user error. If your portrait is broken then a change was made without saving it to the portrait, presumably without you realizing it. A common culprit is accessories since they can be shared between jobs differently than the main gear is. Make sure to double check your glam plates for conflicts. If you have for example a ring saved on one plate and nothing on the other it will break that second portrait when you switch between them unless you use an Emperor's New Ring for that slot.