Unless you've become a paladin refused to use shield oath, I don't know why they would complain. You said you were keeping aggro. I assume there's more to the story? Otherwise, they're just being jerks