You can withdraw them with the command ability located in your companion window from any location, including cities. But yeah, not from the party list.