It's SUPPOSED to be annoying. There's no reason you should be able to just out-run enemies with minimal to no penalty.