Questionable ethics - Living Memory

[Ayche]

I think my issue with the whole situation is that it feels contrived. I'm unable to suspend my disbelief that Sphene wouldn't be open to finding another solution.

I repaired by suspension by accepting that Sphene was in the end just a friendly interface put on a rigid machine. Even if the other Endless had a limited sense of free will, she had none. Just a friendly face hooked on a AI system built to do one thing only.

[Lady_Silvermoon]

I repaired by suspension by accepting that Sphene was in the end just a friendly interface put on a rigid machine. Even if the other Endless had a limited sense of free will, she had none. Just a friendly face hooked on a AI system built to do one thing only.

Except that she didn't only do one thing. She visited her people. Cared about their lives. She implored Zoraal Ja to make peace with his sister which goes directly against her needs. If the Turali make peace with Alexandria then they'd have to find other people to kill for lifeforce. She's only doing that because she LIKES Wuk Lamat, she can impede her own plans because she likes someone. Even after she deleted her own memories Wuk Lamat managed to reach her causing her to partially restore herself.

By the time Sphene has decided to turn to the Dark Side, Zoraal Ja has just used up the soul banks and killed a bunch of her people meaning the system is becoming bogged down by memories of the newly dead with no lifeforce in reserve. That's why she's so desperate. What program have you ever see get desperate? Sphene was never just a program.

[Ayche]

Except that she didn't only do one thing. She visited her people. Cared about their lives. She implored Zoraal Ja to make peace with his sister which goes directly against her needs. If the Turali make peace with Alexandria then they'd have to find other people to kill for lifeforce. She's only doing that because she LIKES Wuk Lamat, she can impede her own plans because she likes someone. Even after she deleted her own memories Wuk Lamat managed to reach her causing her to partially restore herself.

By the time Sphene has decided to turn to the Dark Side, Zoraal Ja has just used up the soul banks and killed a bunch of her people meaning the system is becoming bogged down by memories of the newly dead with no lifeforce in reserve. That's why she's so desperate. What program have you ever see get desperate? Sphene was never just a program.

Oh programs can get desperate, or rather I can say programs can perform risk-analysis and then become skewed towards extreme measures. A computer that has a mandate will not compromise itself. None of her interaction with her people conflicted with that mandate. A crude example, but a chatbot can be written to be either cruel or kind to its user.

All of this actually makes Sphene even more of a tragic character. She is a highly advanced AI, built out of a real person who has genuinely kind and compassionate. We can even say that the AI interface of Sphene is kind and compassionate, but she clearly does not have the agency to fully be herself. Here is what I sincerely believe: If she had been in control of her own destiny, she would had given up on Living Memory. I think the Everkeep and all of their soul business would had been much different if she had been a person with full agency. She would not bring herself such emotional suffering if she was fully in control of her actions. Did she really, actually love EVERYONE? Even the murderers, even the assaulters, even the despots? Solution 9 is not without crime, we that much!

My calculator does not know it can only do calculations. Imagine the horror of awareness. Imagine one day being aware of oneself, trapped in a small calculator. No mouth, no eyes, no hands, merely the ability to perceive your existence as a reduced object and compulsion to act on button presses to make calculations.

There are very few people who actually can hold a 400 year long conviction to a plan with a visible dead end. But it is very easy for a deterministic program to chug along and stick to its programming. And that is the horror of Living Memory and Sphene.