It's kind of annoying when you're gathering, but that's what stealth is for. If you're a high enough level in relation to the enemies around you, you can stop to take in the sights at whatever pace you want as long as you're invisible.