I have the same issue. It did not start until early access. It seems to be totally random for me, but it has been happening more and more frequently as of today and yesterday.

I really hope someone addresses this soon, its preventing me from enjoying this game.