Correct me if I'm wrong but I can't recall a single time we've been able to teleport to a location that isn't loaded on the servers by Default. For example you can teleport to your house exterior but not the interior as the Wards are always active but the interior is only activated when somebody enters it (and for the same reason you get booted out to the garden after a duty or logging out).

Pretty sure not being able to teleport straight there is a tech limitation as a trade off for everyone getting one of their own.

Hopefully they might at least consider adding it to the duty finder menu as a way to enter from anywhere.