Complain when it causes a problem. Players will respect you more for it.