That was the case for OP also.

It's not the discords they have a negative opinion of. It is raiders in general they have a negative opinion of, whom they incorrectly assume are "elitist" and "exclusionary".
I truly get what it's like when you've never raided before - you think that raiders are some elite bunch trying to exclude you. But then you overcome the anxiety and join a party. You go through hours, days or weeks to get that clear. Then you don't want to go through that progression again, so you make a [Duty Complete] party. Suddenly, you have become the "excludes people" bunch. But once you get that far, you realize there is no other way; you can't go through all that prog every time just to reclear.
But the game is actually so large and cross-world now, that first timers can often find a full party of people new to the content. So it works out anyway, and everyone is happy, except those who haven't overcome that anxiety to try joining a High-End PF or discord.
So yes, a lot of people will naturally think these discords are some scary elite bunch. If they garner the courage to join a run their opinion will quickly transform.