This was shown in the echo flashback after the aery.
Of the twelve, four died. One became the founder of 'The Forgotten Knight' as he and three others walked away from the whole messy betrayal, stating they had sworn to follow Haldreth, the son of Thoradan, and not Ishgardian as a whole. This left the four Knights of the four great houses to be stewards of the empty throne. Haldreth was the first dragoon, it seems, or maybe Thoradan was. He chose to abticate the throne, seemingly because of the betrayal the elzen committed against the dragons.