Keyboard and controller are character settings so you change them only for your character. And them are separate afaik, if you switch mode you find them as you had set last time.