I don't like it either, but honestly, unless you're doing Savage content, just use them anyway and who cares if you're a bit less efficient.
That's SE's problem, not yours.

Now sure, if you're doing top end content, you want to be the best you can be, so you'll want to have keybinds to your keyboard, period.
Not to a controller.
But for casual content? Just go with what's comfortable.

I main WHM, so I don't use macros and set everything to my Keyboard for party content ... because I kind of have to, you know?
It's the trade off I make for not having to wait in line.
But if I'm going DPS in casual content or something, it hardly matters so long as I'm doing my job well enough.
...and for FATES? I just as well lost my keyboard altogether xD