I've got 29 and I think some were venue spammers that were particularly obnoxious about it. A few were a premade where my co-healer bailed after the second Ridorana boss and they refused to kick them, making me solo heal through the next two and when I mentioned the vote kick failed in alliance chat, one of them went "Snitches get stitches" in party chat. Their friend magically came back just in time for the final boss. So they'll stay on that list forever while I'll probably clear others off since those characters may have moved on by now.

I don't really have judgement if someone has a lot of people on their list. They just have people they don't want to see things from so they're using the in-game tools.