Doesn't he say he doesn't try and kill Minfilia (or us at the time) because Elidibus wants us alive and he doesn't want to have to put up with his whining? I think he doesn't kill Minfilia for that reason. The only reason he tries to kill us is because we basically follow him and attack him.