I agree with a penalty, but it should be no more than an hour; at very least until vote kick is instated. I know it may be hard to fathom, but occasionally you do join in on a party that is incapable. I have only ever been the first to leave once (the second a few times), but I know there may be a time I have to do it again in the future and i'd hate to lose out on 24 hours because I had a bad PUG. Now, when vote kick is enabled, sure. Because then you'll be able to manage your party.
That said, at level 50 I think it's better to be pulling from your FC instead of the DF. I am only just getting ready for my first 8 man, but it looks like the coordination provided by playing with friends and FC members will make it way more fun.
