It would be nice if you at least got to see how long you had left before you could call trusts.

There's a workaround for this if it's a problem for you specifically though: the player who invited a new party member is on Trust cooldown, not the party. Pass lead to somebody to invite a new party member, then have them pass lead back and you can pop your NPCs immediately.