The only way to permanently remove the sprout icon is to play the story. I believe the first quest in Shadowbringers will knock it off. Good luck.