This is just server lag.

Give it 1 second before casting your spell after swiftcast, wait until you see that "+Swiftcast" image coming up your char, and it will work