Digital Foundry did a comparison of various versions of FFXIV and found the native PS5 version to perform worse than the PS4 version running on the PS5.

https://www.youtube.com/watch?v=trIiUXdEyo0

PS5 comparison starts at around 28 minutes.