You never considered that maybe there was just a lot of healers queueing for Library at the time you were? Occasionally i have to wait some minutes on my Paladin for Aery, Sometimes my Monk gets instant Vault queues.

Just being a Healer does not guarantee you a fast queue all the time.