That does sound strange. What OS are you using? 32 or 64 Bit

Have you compared Full Screen and Windowed?

Are you running all the default settings in NVidia Control Panel?

If you have the Max Buffered Frames set high with VSync on it will really bog down the GPU but will give you very smooth frames. In your case it might be using up all the vRam, but not actually pushing the GPU very hard.