For me, that distortion occurs during the process of streaming to an online service (Twitch, in my case) but not when recording locally. So if you're doing the capture and upload as one step, you can try separating the two if that still suits your wishes.

Beyond that, given that Shadowplay uses Mb/s for its units, even that 30 is generous overkill for what should be top quality at 1080p60. Twitch doesn't even support beyond 3.5.