Thanks Lori and Brian, I totally forgot there was a gamepad shortcut to dismount. (I recently switched from KB).

On a sidenote, it seems you cannot disable the mount skills hotbar, such as like you can with the pet bar. It switches to a crossbar that is separated from the R1 button so tring to tap/hold R1 doesnt work.

But if a dev reads this, it could be handy to still see your original cross bar, whilst mounted. If you really wanted to use a "vanity" skill, you can just press it on your mount/pet skills bar. It seems a waste to have it in the mount skills bar AND it overwriting every skill you have in your cross bar.