Game is suddenly very laggy after installing new gpu

Just yesterday I upgraded from a GTX 970 to a RX 6700 XT
I used DDU to completely uninstall the Nvidia graphics drivers in safe mode, and then installed the AMD Drivers opting to only install the drivers and not their Adrenaline software.

my current issue is that when I launch FFXIV I get max maybe 20-25 fps whereas before I got 60-70 pretty consistently except for in Limsa.

And I know this is only a FFXIV issue as I am getting upwards of 122 FPS in Cyberpunk2077 on Ultra settings and the only thing I've upgraded was the gpu

So, I was just wondering if anyone has had these issues as well and was able to fix them.

specs: Intel Core i5-11400F
ASRock B560M Pro4
AMD Radeon RX 6700 XT (Navi22 XT) [ASRock]
16 gig ddr4 3200 OLAY