Yup. It has to be built within 45 days or else you lose the plot. It follows the same rules as if there was an estate built; you get 80% of your gil back that you collect from the caretaker.