My FC is strictly MALE CHARACTER only.
Apart from very slow initial recruitment, we had no problem like receiving hate or insults.
Sure, we had a few people chuckled at the idea but we also received support from people who likes it but not willing to depart from their FC.
(The FC has about 20+ members now plus CWLS for cross-world members)

Perhaps you should reflect on your recruitment approach.