Potsherds for the mount seem to drop from Bronze chests, as well as a single one you automatically get for finishing each 50th floor.