That's because Bard is a 2.0 job. They're available without needing any requirements. Once you choose a job stone for a retainer you cannot switch it.

The only thing wrong you did was assume that you can switch from one job stone to another job directly without resetting back to level 1.