Quote Originally Posted by Jojoya View Post
"Hey Bob, you done with what you were doing yet? We're at Gaius and ready to kick the rando healer from the group so you can join to get your tomestones."

That is why you can't join in progress.
A simple fix would be to make it so empty slots in duty finder groups can only be refilled by fetching random players via the duty finder, and disallow inviting specific players. That's what WoW does, I think.