Give it time, we just liberated Ala Mhigo so the city is still recovering from the battle. That means certain areas are restricted including the Castle where the Royal Menagerie is located.