I'm fairly certain that "up/down arrow" you're talking about is assigned to shift up/down (or is it ctrl or alt? it's one of those, anyway). I've definitely triggered it on accident, so it's definitely something. You might be able to hotkey that to something else...?

The only other thing I can think of is that you could technically put a macro on a hotbar (that can be hidden), assign a hotkey to that hotbar slot, and just use the macro to swap the contents of your hotbars.