This is actually answered to a degree in the lore book. I can't remember the exact passage, but it states that soul crystals are actually a huge drain on a person's aether due to the attunement required. One soul crystal is safe for most people with 2 being possible amongst the highly aetheric. The pc by contrast generates a whopping amount of aether by way of the echo or blessing of light which is how Anima and Fray could draw on them to sustain themselves temporarily.
It should be noted that job crystals get more aetherically costly as they get older, the MCH one for example doesn't have any aether drain which is why the less magically inclined can use it. It should also be noted that whenever the pc swaps out of a job, they temporarily unattune to it as evidenced by the level 80 drk journal.

many adventurers and even none adventurers train in at least 2 classes as we know them, so classes by themselves are not too strenuous- they just take time. Why else would so many guildmates cross train in conjury, or even Thancred having rogue and gladiator training pre gunblade.

it should also be pointed out that the WOL at least in previous iterations of the game could use multi class abilities before SE sadly removed them.