Once you finish your lvl20 main story quest you can join a grand company.

Once there, there's a quest called 'my little chocobo' that you have to complete to use mounts. 2,000 company seals are needed to purchase the chocobo insurance ^^