Just left click on the rocks. That won't make you deselect them.