Sometimes being a 'Little Lady' means having to fight off those unwanted followers