It still happens if you manually equip your job crystal from the character sheet. Doesn't if you just use gearsets though.