The biggest problem with charm is that really you just couldn't use it in some places. BST could only be effective in the open world and was virtually useless in 99% of BC fights, which didn't allow anything to be charmed in 99% of cases because charm would trivialize the fight (what easier way to win than to make each of the mobs in a BC fight attack each other?). It also depended on an area having mobs of types that make sense to be charmable. The shift in focus to jug pets was most likely more meant to allow BST to be useable everywhere instead of only here and there. You say people didn't care if they didn't get invited to events, but you're kidding yourself if you don't think people would have enjoyed their favorite job more if they could use it to get things done instead of it being more a novelty (or troll, in some cases in the early days

).
As fun as it was, to have the whole job revolve around charm was not really the best decision from a good game design perspective. Competing for charmable targets was common in the early days. There were lots of problems because of this. It wasn't a sustainable design, as any time BSTs got near eachother or other parties there was inevitable disputes, which typically ended with one MPKing the other person or party.
Also the "stat vomit" or whatever you ant to call it has absolutely nothing to do with this "job soup" thing you're talking about. It exists purely to scale us to being pseudo-level 119. it by itself does nothing to make jobs more or less similar.