You'll find the blacksmith guild in Limsa Lominsa, along with the Rogue guild that you also want.
Crafting is its own activity with its own skills. There are skills for progressing a craft, skills for increasing your chances of a High Quality (HQ) result, skills for increasing the success rate of other skills, and so on. Like the combat classes, it starts off very simple and then gains complexity as you gain levels. Most of those skills will be the same for any crafting class, however at levels 15, 37, 50, and 54 you gain special skills that can be used cross-class. This means each class gives a different skill, but once you advance that class far enough to unlock the skill, you can use it on other crafting classes as well. This pattern makes your crafting more effective if you advance all the crafting classes to unlock each of their unique skills rather than just picking a favorite or two. (Note: most of the important cross-class skills come at levels 15 or 50, but Culinarian gives a really good one at level 37.)
So while blacksmithing is a good place to start out, and decide whether you like the game's crafting system or not, if you do decide you like it, then you're going to want to pick up the others as well. The first 15 levels will give a pretty good idea of what it's like, and it's also the point where you pick up your first cross-class skill.
I do recommend that everyone at least try out crafting that far. If you decide you like it, unlock the others and keep it up. If you decide you may like it but not enough for the time it takes, come back to it later once you reach endgame or whenever you want a break from combat. If you decide you don't like it, there are plenty of other ways to get good gear (including buying it from other players who do craft), so it's not a problem if you decide to drop crafting entirely.
EDIT:
Crafting and gathering involves a lot of materials, and if you try to get too deeply into it before unlocking retainers, you'll run out of inventory space. It's ok to start the first few levels before then, but try to get to The Scions of the Seventh Dawn in the main scenario before you get too deeply into crafting, so you can get retainers to hold excess items and to sell your stuff.