My suggestion would be to use WASD instead of arrow keys.
I've done some mild testing for you and it seems there's no way that I've found to resolve your issue.