Any time your chocobo isn't actually standing beside you, the timer stops. You can check how much longer you have in the "Companion" menu.