It could be possible you were too far from whoever said it in /say, and there were tells as well from what I've heard.

Either way, it still does not excuse the behaviour that continued afterward. He did not join the next duty after that one, I never joined one until after that entire party left. However as I crafted I was getting harrassed. "Are you two fags leeching our caravan runs?", among other comments. Inexcusable.