That's not really 'brainwashing' so much as it's a legitimate concern. Even assuming that they did surrender there is very little guarantee that they would be treated well - and even if they were kept alive it would likely be indefinitely as prisoners. Plus if someone as supposedly 'pure' as the Warrior of Light can slaughter rank and file soldiers - even with the knowledge that many are just conscripts - and forcibly tear out their souls to empower their weaponry then it doesn't likely give much incentive for them to believe that the fate in store for them would be pleasant.
The nearest city to Castrum Meridianum is also Ul'dah. A place that has been, uh, 'culturally enriched' by many Ala Mhigan refugees. Many of whom have joined the Immortal Flames which, presumably, would boast a fair number of them. Given how irrational many Ala Mhigans have shown to be in the name of their homeland it's easy to assume that there's a genuine risk for atrocities to be committed where Garlean prisoners in their 'care' are concerned. If it's the Brass Blades that are in charge of prisoners, however, then not only is there a similar risk (I suspect it is also full of a good number of Ala Mhigans) but we've seen countless examples to show that the organisation is prone to corruption.