When I first opened my client (the window that updates your game, where you log in etc.), it asked me whether or not I had a Square Enix account. I had one from open beta, so I simply used that to log in. Then, it asked me to enter the CD-key that I got when I bought the game (found inside the case). After entering it, it automatically detected that it was a Collectors Edition and showed me a list of items I would receive. After logging in, all 6 items were in the mailbox.

Note that when you use 4 of the items (mount, 2 pets and an armor for your mount), they will disappear from your inventory - but fear not, they are simply added to your character. Note though that you have to be lvl 20 to use mounts, although pets can be used right away.