Unlock either Botanist or Miner. They can both gather all shard types right off the bat at level 1 (once unlocked, you can check your Gathering Log for where each type is found). The other way is to buy them off the Market Board. You'll eventually need a lot, so even though they're quick to gather and cheap to buy, it turns into a lot of either time or gil once you start dealing with high volumes.
At level 19 (any class) you can unlock the ability to convert used gear into materia by taking the quest Forging the Spirit.
At level 19 in a crafting class that can make gear (so any craft other than culinarian) you can unlock the ability to meld materia onto another piece of gear by taking the quest Waking the Spirit.
Then at level 25 in a crafting class that can make gear, you can unlock the ability to add more materia onto an item than it has slots for (up to a maximum of 5). This is unlocked by the quest Melding Materia Muchly.
Besides unlocking the ability, there are class/level restrictions on what type of gear you can meld materia onto. The same crafting class that makes and repairs a particular type of gear is the one needed to meld materia onto it. So just because your leatherworking class is high enough to do that quest and unlock melding doesn't mean you can meld materia onto plate armor if you don't have armorer or onto cloth armor if you don't have weaver, etc. (This restriction applies only to the melding step, not to the converting step, which any class/level can do.)
As mentioned, most people don't bother melding materia onto low-level gear due to how fast you out-level it, but it's still useful to learn the ability to convert used gear into materia. May as well turn some of that old outleveled stuff into something valuable. (Also, though most people don't, I actually did meld materia onto a few pieces of mid-level crafting gear. Not really necessary, but a nice little boost, and even though it's only useful for a narrow level range, the fact that I'd be taking eight classes through that level range meant it's useful quite a while before outleveling it.)