That's been brought up several times already. Fighting outside Ballista is a nice idea, I would want it available as an option. As for the penalties, I agree it sucks, but the fact that you blink when you swap gear isn't the issue. The issue with that is when you blink, the person fighting you loses target from you and would have to disengage to reset it, otherwise it is a real pain in the ass for them to use abilities on you. Sounds nice? Not really, people would exploit that all day if they didn't get penalized for it. So no to gear swapping still. Unless they fix blinking without using third party tools it'll always be this way.
However, if there was an option to duel outside of Ballista/Brenner, there technically probably wouldn't be any set rules other than you can only attack the people involved. I imagine gear swapping might be allowed with that, but you never know. It's SE.