On a PS3/PS4 controller, that indicates the item you'll harvest. You use up/down buttons to set it to the item you want and then X to activate the mining action. I assume it's comparable on PC with keyboard/mouse. A player who prefers the mouse could click items directly (so wouldn't really need a selection indicator) while a player who prefers the keyboard could use up/down arrows to select and enter to harvest the selected one.

(I'm speaking from knowledge on the PS3/PS4 part of that comment, but from conjecture on the PC part.)