Sorry for the bump, just want a reply here before a mod closes it.

The issue can get really annoying, specially when crafting or gathering. Any small nudge of the mouse makes the controller lost focus completely.