I have never experienced this. Swiftcast has always granted my next spell an instant cast. Are you sure you are using Swiftcast and not Surecast?