Your graphics card will likely want a PCI or PCI-E slot, with the latter being the more advanced and better option. They're also distinctly visually different, so if your board has a mix, you'll be able to tell. The case the PC came in may also have detachable shields in front of these slots, which is usually easily handled with a screwdriver and just popping them out so the connection slots can poke through the back.

If you are just using an on-board chip and do add a completely new card, that will be a guaranteed power supply increase relative to the power of the card in question. There are some sites out there that'll let you punch in your parts configuration and estimate the needed power, so google that later if you're running on something like a 250 or 300W PSU, because it's more than likely going to be insufficient. Aside from the card slot, you'll also need a specific connector to hook it up to the PSU. These will widely vary and some cards are kind enough to come with adapters, but you'll see stuff like 3 or 4 pin, stuff that looks more like a plastic USB port with 2 pins, and so on.

Frankly, changing the PSU can also seem pretty scary due to all the connections you'll need and the possibility you may have to drill new holes into your case to secure it or cut a slot out for the breaker. After a point, if this starts looking like a project beyond just getting a new card, I'd look up a local PC guy and let them handle the process.