Sounds like my workplace.

But yes, I can see where you're coming from. While I haven't been an integral part of anyone else's FC except my RL friend's, which I'm still a part of, I can say I've seen that sort of behavior in more public spaces, where many people congregate to idle and chat, craft, or peruse the markets. I've been part of a treasure hunt group where I learned the leader was angry at me, yet no arguments were had between us, no disagreements, no hostilities--nothing. I was simply not part of that person's friend circle, and to this day, I'll never know what I did to make them so angry. They wouldn't talk to me. None of their friends would. It's very much like high school, but worse. It's worse because a lot of us didn't give a damn about high school, but we all care about FFXIV (to an extent). Some of us are more social than others.

At the end of the day, I feel like the only way to overcome it is to--if I may also quote something I enjoy, it's from a band I like--"take yourself to higher places."