Yeah that is true. It's probably why there is only two dialogue options. Plus the WoL has had to deal with, at least up until shadowbringers, beast tribes that were fragemented into factions. The first faction, the ones the WoL helps, the good ones that don't want to cause harm unto others and as you said want to change the system for the better and don't want anything bad to happen to them. And then the second faction, the zealots who been brainwashed by primals, who often seeked to harm others and enforce brainwashing onto others. The only beast tribe that doesn't follow this rule, at least for it's more zealot side, was the sylphs, and that was because Ramuh himself also behaves differently compared to other primals, much more sensible than those like Titan or Ifrit.
So that might have been the original intent of the role quests in dawntrail. That the good tribes you encounter that been robbed are the ones would have helped along the way, while the villains are like a microcosm of the more zealot side, and each totem does contain some powerful beast inside that is basically a primal in power.