I have always had this issue. I have to keep all my most important abilities together on the 1st action bar because scrolling between them is too difficult. Basically it feels like I have 10 abilities equipped rather than 30. It needs fixing urgently.

What I do usually do when the up and down arrows dont work is I just press a number and then press up or down, and then it generally works.