Good question, but here a reasonable answer - gettinggon top of the tower from the outside might be impossible since it could be surrounded by a magical barrier that prevents entering or causes any flight mechanism to cease working.

Think about Azys Lla - without the key, we had to brute force ourselves in, with help from major plot characters.