Something to consider if you're using current gen console controllers, the micro USB type connector on them are notorious for coming ever so slightly loose after being used a while, causing intermittent loss of connection sometimes if you're connecting by cable. Took me years to realise that was the damn issue. If you use XBO or PS4 controller wirelessly then there shouldn't ever be an issue, as long as you're not experiencing wireless interference or driver conflicts.