Elidibus can't be that smart if he thinks nurturing 5 new Warriors of Light are going to be able to defeat the Warriors of Darkness. We are 7 times rejoined with the main hero being 8 times thanks to Ardbert, so he would have to nurture the new Warriors of Light back on the Source if he wants any chance of us being defeated as well as manipulating their minds to think we're evil. Thing is everyone on the Source knows we're not evil, we've done so many good deeds and bards sing of our adventures to spread word around, so it would be very difficult for anybody to not know about us unless there's another land where we haven't yet visited where word hasn't yet got around.
The other thing also is that the Warriors of Light have never struck anyone down without due cause. The ones on the First refused to kill the Shadowkeeper as did the main hero when she asked to be struck down, and when they came to the Source as the Warriors of Darkness the hero didn't kill them. I would vouch the very same thing would happen with the new Warriors of Light if they were capable of defeating the Warriors of Darkness and I'm pretty much sure Elidibus would know this too, so he must have something else up his sleeves that we don't know about. Elidibus would probably do whatever it took to avoid direct combat with us unless we were at a disadvantage since he is only an emissary and the last of his kind, if even Emet's full force against us still resulted in his defeat then for Elidibus to go head to head with us would be suicide.