It used to work with earlier versions of Monterey but 12.3 removed support for Python 2, which stopped FFXIV from working entirely. They are working on an update that might release next month if testing goes well. There is a player created fix at xivmac.com that will work right now with the latest version of Monterey, although it is only compatible with M1 or Radeon based Macs (although it does offer optimisations for M1 too). If you run Intel HD/Iris or Nvidia you should wait until July and hope that works with your chipset, or just install Bootcamp and play the Windows version.