While he was rude, and it wasn't called for, telling you that was not harassment so they wouldn't really do anything about it.

(To play Devil's Advocate) Based on that, they could see your calling for the party member being temporarily AFk harassment. They could be on VOIP and the person needed to leave for just a few min but you were so quick to get rid of them because they had to go or couldn't play with only one hand while eating a sandwhich. It's no different than people early pulling and leaving someone behind.