Been having the same issue. Will go 1-2 days without a problem and then it starts happening constantly. Can't pinpoint what triggers it, whether it's been the controller, keyboard, or mouse, or if it's just randomly happening. All do not work once it's "frozen". Someone on Reddit reported a similar issue and said switching the USB ports helped, that hasn't been the case for me though.

Also have not experienced this on the PS4 version on PS5.

Hope this is resolved at some point. :/