What's your setting for: Character Configuration --> UI Settings --> General --> Restore map when not moving? That mentions moving rather than interacting, but it's the only relevant setting I can find. If you have that turned on, then maybe turning it off would help.