You're directly affecting other players by leaving a duty or not accepting a Duty Finder that you queued for. They had to wait in queue also and now you're making them wait for a replacement on top of that. If it's happening that frequently then you should probably consider why rather than being upset by the consequences.