Unless they say anything in chat to indicate so, you can't prove they're griefing, they may have just been disconnected or forced to afk for whatever reason.

The timer is there for a reason, waiting for it is no different to waiting for a queue or whatever.
If those few minutes are worth more to you than the loot, or vis versa, that's your choice.