For me the last time I had this issue was some audio software causing the device manager refreshes. Maybe it was periodically checking for headphones or something, I don't know. Took a while to get there though, I tried disabling everything one by one until I found which one was the problem.