I think it was mentioned that those who can potentially manipulate time are very rare; so time magic is not really inherent in all Astrologian and the only one known, thusfar, is the head of the Bibliothecs.