Had the same problem with the exception that I didn't have to open the task manager. I just right clicked the ffxiv tab in the task bar and closed it. I also noticed this only happened to me when I logged out/exited the game when not in a sanctuary. May have been coincidence but worth noting.