Your retainers will still be there if you stop paying for them. You'll just be unable to access them as their contract is "Suspended".