But you literally can't teleport another person with you to your estate, try it. You simply can not.

Outside of that I don't know what is causing your issue. Try not sitting down or don't talk to an NPC as I don't think you get a message for that until you miss it.