Stuttering with a GTX 670

[Mr. Plankton]

i5 2500K @ 4.0 GHz
GTX 670 Power Edition
8 GB DDR3 RAM @ 1600 MHz
1TB HDD @ 7200 RPM, SATA III

Currently on driver version 310.64. 310.70 and other driver revisions did not fix this issue. Game is up to date. All settings at max, except SSAO samples is set to 16. V-sync is on. Temps are normal, etc, etc.

I've been getting stuttering while playing this game, especially when completing actions (using the lab equipment to make the serum for Agrippa, getting damaged by the shadow's blobs, walking into the room and Agrippa starts talking, the shadow "reveals" itself and that weird sound effect plays, etc.). Basically, when something like that loads the game stutters for a brief moment. It's annoying not being able to maintain a solid 60 FPS when my GTX 580 had no trouble doing so whatsoever and without the stuttering as well.

I also notice that without "prefer maximum performance" enabled, the FPS will drop to 30 on occasion, even with triple buffering being forced on through the CP.

Any ideas on what to do to solve this? I've tried lowering settings, removing V-sync/trying adaptive V-sync, using an FPS limiter via NVIDIA Inspector and none of those things have worked.