I think I found a fix for full freeze issue, I rolled back my nvidia driver to 314.22 and it works, but instead of freezing, the driver will just hang for 5 seconds which is still doable.