I just verified that there is an aggro-free path to the Mahogany Door (and the Grounds Tome near it, which makes 3-2 doable). Just take the southern paths and you'll only pass non-aggressive giant bats and beetles.