I didn't really mind, also they already start with tearing down the tower a few weeks earlier, while you're still in the garrison "stage"

But it was a bit ironic if you compare it to the Ananta beast tribe quests, since they have a similar setting of Ala Mhigans wanting to tear down and rebuild the big bridge and castelum over the Velodinya river. Except there we have to help preventing that as it was a horrible idea that would've crippled trade to Ala Mhigo