The current penalty is fine. The real issue is how long it can take to get a party started, or replace a leaver.

There's a serious lack of tanks, always. Healers are also lacking but not as badly. The incentive for being an "Adventurer in Need" needs buffed significantly. Gil is worthless.
There's also NO WAY to know a party in progress needs a replacement, without queuing for them.