The idea has merit but I think ultimately it depends on a number of factors.
How often does the Hamlet Defenses occur? Once or twice a day, no problem. Once or twice an hour (24 to 48 a day!), no way. An unpopular hamlet would always have its buildings in a perpetual state of disrepair which would drive people away from it and to the more popular hamlet.
Something else that would influence this is how much damage could a building sustain during an successful/unsuccessful Hamlet defense? Could it go from 100% to 1% in just one battle or over the course of a week's worth of battles? So, are you constantly and endlessly repairing buildings with little gain or is it something that needs a bit of player input every once and a while?
What could make it more successful is if the repairs were available through a bunch of leves (ones that are scaled to your current crafting leve so you're not crafting something 20 levels below your current level for a little to non-existent reward) where each crafting class has a list of items needed to help repair the hamlet. Even better would be if these leves could be picked up at the Adventure's Guild from each city the hamlet is a part of.
And to round out the leve experience, new leves get offered to deliver the goods to the Hamlets. These leves are part chocobo race, part dodging monsters. Every hit results in dropping or damaging some of the goods being delivered.