Partly a meme, partly rose-tinted goggles, partly some people just like to have something to complain about, and partly some people just liked it.
And I don't mean that to be derisive - some people genuinely liked it - but mechanically, it was like PLD having Fight of Flight and Requiescat. You never used the second without the first, so it didn't bring anything to the table. It also made balance and the auto crit stuff kind of weird and devalued crit and stuff (this was before they implemented the "auto-crit/direct hit abilities do more damage based on crit/direct hit effects" thing, which would have helped with that). It was slightly more involved since it was tied to pooling a resource, but in practice, it was kind of bland in the "you always use these together, why have a second button?" way.
But some people really really liked it. I've also seen very rational and sober minded theorycrafters and guide makers say that it really wasn't all that and explain why they thought so. So both sides have a decent argument to make. (It's funny, I still haven't leveled SAM...but I still have the grayed out version on my bar for posterity.

)
You're right, though, that SAM feels very good right now. If you never played with Kaiten and you do like current SAM, don't feel like you have to hate it or should "join the chorus", just keep loving the thing you love. Nothing wrong with that. : )
EDIT:
Believe it or not, the complaints are not unanimous.
Again, there are theory crafters, guide makers, etc that can give reasoned arguments for why Kaiten's removal was not a bad thing/was a good thing. On other forums (not this one for some reason), the position is not unanimous. Reddit, for example, but r/ffxiv and r/ffxivdiscussion (which is more technically minded people discussing mechanics and stuff) has dissent against Kaiten being a good ability.
I'm not saying one way or the other - both sides, imo, have good arguments - but I am saying complaints are not unanimous.
.
Oh, and don't get me wrong, I don't know that removing Kaiten WAS a good idea. I'm just pointing out there is more discussion/disagreement on the topic than skimming this forum would suggest.