It is very annoying to be enjoying the BGM for a zone while doing FATEs, but as soon as I enter the FATE I'm teleported back to ARR with terrible FATE music that overrides whatever it was I was enjoying.

There is no option to mute FATE music and listen to the zone BGM. The only option is to turn off ALL BGM. This turns FATE grinding into a mind-numbing activity, since there is no gameplay happening other than pressing my AoE button and move out of the orange line on the floor.