Their existence in lore is established, but their avaliblility in lore is probably much different than in our video game. Those bandits you mentioned might not have the skills to make glamour prisms themselves or the money to afford them.