People accidentally throw away irreplacable items all the time. I'm pretty sure you can throw those away if you really want to? I haven't tried, but I don't recall the game preventing you from throwing away anything that can go in the inventory unless it's equipped or placed in a storage space other than the inventory.
Aside from that though, I absolutely approve improving item storage solutions.