Furthermore, advanced classes require a Soul Gem in order to switch to, crystalised memories of previous members of that class. To become say, a summoner, without one is absolutely impossible.

In XIV lore, it is stated that Eorzeans do have a magical capacity (something Garleans lack) and I'd assume, as Orophin stated, that switching to a conjurer is plausible should the character have adequate training from guilds - as well as the capacity to do it themselves.