I've wondered that too! Any time you see them you're just holding it, not attached to anything.
The lorebook just says they are worn "close to the skin".
My best serious guess would be that you might put it in a pouch on a necklace.
I wonder if you have to be careful with handling multiple soulstones at once?
