Hmm, I am aware of some issue that Nvidia's Game Filters/Photo mode are having with games that can cause FPS Drops/Stutters.
If you have the Nvidia app installed, maybe see if turning those options off helps?

I use an RTX 4070, and I had small frame drops that started around mid/late December, a little after 7.15 dropped, but turning off those options in the Nvidia app helped there.
Nvidia's working on a fix for the game filter/photo mode FPS drops, last I checked, but fingers crossed you've found some fixes for your situation.