You can though for most items iirc. You can do so at the Calamity Salvager. For example, if you have a yukata for females, you can exchange it for a male yukata. Same for wedding dress/groom outfit, and more.