I thought that too. But remember all the way back in Stormblood. How he "lived" for the moments of sparks flying. He didn't care who gave it to him as long as someone did. If so then it didn't matter if it was the WoL or someone else who is potentially as strong. The reason I believe he focused only on the WoL was because he saw the WoL as a friend.
