That was indeed the case and infact was more the reason why Ishgard has started tearing itself apart following the revelation of the origins of the Dragonsong War - as Ishgard's theoracy had made up that only those of 'noble blood' were descended from Thordan and the Knights Twelve, where in actual fact, every Ishgardian, either noble or low born, traces their ancestry from those six remaining knights (remember, there was actual six surviving knights not four as shown in that Echo scene, when the survivors decided to "leave the throne empty" and establish themselves as stewards, two decided to cast away the sword and settle down into "honest trades", one of whom strongly hinted to have been the founder of the 'Forgotten Knight' pub in Ishgard based on his "serving ale instead of steel" remark - hence the pub's name rings true. The other one, it's unclear where he went or what he did, he had no dialogue and wasn't even named in that Echo scene. So those two nameless knights presumably were the ones who established the 'lower ranks', the common citizenry who eventually became downtrodden as the 'lowborn'. The remaining four knights were the founding members of the Four Noble Houses of Ishgard and established the city's nobility, eventually intermarrying so much that the actual family lines are probably now very blurred indeed.
Thus this alone probably caused more anger and resentment among Ishgard's citizens than the revelation that elezen and dragon once lived together in harmony or that it was Ishgard who betrayed the dragons and caused the Dragonsong War, which is sadly also partially the reason why Aymeric's reforms to restore relations with the benign dragons of Anyx Trine are constantly meeting with resistance from his own people - because it was the nobility who lied to them, it's ironically caused a situation where to many in the Foundation no matter what a noble says now it's bound to be a lie, and thus reacts accordingly. It's a sad situation to be true, but something which can be overcome. Alas, with Nidhogg's shadow continuing to darken the skies it will be a long time yet before Ishgard and dragonkind finally sees peace.