I'd suggest just toying around with the different settings and seeing if one of those fixes it.

The thing that ended up curing my FPS stutters was reducing light reflection via Effects > Glare > set to low. Given that almost every object in the game is subject to some degree of light reflection it can make the FPS extremely variable depending on how aggressively a given area makes use of it.

Could also explain why it seems to trigger on skill usage due to lighting being involved in those, too.