It seems like, due to FFXI being an old game and handling some things in odd ways, that it "takes over" the controller input for some reason or another. Although the built in gaming features in windows 10 are a nice addition, you may have better success using other software for recording. While OBS Studio is designed for streaming, it can also be used to do local recordings.

As for a fix, I do have one idea. If you are currently using DirectInput for FFXI's controller inpt, go into the game's config tool and switch to Xinput- Support for this was only added very recently, so it is possible that you are still using DirectInput and switching it may address the issue. (If it is already on Xinput, try switching back to DIrectInput and see if that has any effect).