Tie instant portraits to gear sets, ensure there aren't any portraits set as the default instant portrait.. and it will still often show the default.

I've also noted that you need to reconfirm your gear set to make sure there aren't any weird errors associated with it, even if the set is current, or it will fail to load.

Am I doing something wrong? Or is the system in need of a few bug patches?