The whole point of an invite styled system would be to avoid abuse. There's nothing stopping a group from vote kicking and attempting to allow a friend to join as things are right now anyway.
Someone disconnects=>invite friend or lsmates=>they dont even que, they just click accept.
Only problem i can think of is that linkshells and friendlists cant be pulled up while in dungeon. You would have to right click their name in chat log to invite or something.