You can't use the ferry from Vesper Bay to Limsa until you get your airship pass. They will deny you passage. You must progress your main story quests until the point where you are given your air ship pass to get to (or leave) Limsa.