If you look at the ride side of the window under the words Appearance then Item, you will see the word "Remove".
That is the Remove button.
Don't feel bad OP, you aren't the first person to not see it.
I was also stumped for a bit at where it was the first time I tried to remove glamour.
I didn't think that was the remove button and spent several moments trying to find the remove button.
Since then, I've answered the same question several times across my LS's.
I really hope they make the Remove glamour button stand out more.