So the thing I have discovered which is not obvious in any way is that if you are wearing gear that hasn't been registered with a gear set then portraits wont work, even if none of the portraits have an error mark next to them. If you do anything to your gear, such as add materia, you need to refresh that gear set and then you need to update the portraits as well.