Seems like it's still sort of random. I can never get back the server group without restarting the client; but there seems to be no pattern in causing it to go.

I am disappointed. But at least I can remove my tinfoil hat