I am having this issue too (and works in other games as well fine) but could you do me a favor to confirm something I been thinking.

Can you keep Computer Management > Devices open and when it becomes unresponsive, have a quick look to see if the controller disconnected and reconnected (it should refresh the device section).