I play with controller and find flying extremely easy.. I'm not sure what you're doing wrong.