I would only kick someone if they're going a million miles out of their way to get it and/or if they're pulling/dying to a horde of monsters on their way.

If I can see the ground icon for a hoard, you can bet your ass I'm going to go get it unless the room is full of mobs.