It's possible you could have the option ticked, where the hotbar automatically switches to the specified one when the weapon is drawn/sheathed.