Honestly, I don't see DPS and healers leaving when the party first forms. I do see them leave in droves after a wipe though. Either way, at least they tried before quitting.

But you're right, it wouldn't be "fair" to impose such a penalty on only the tanks.