@NounVerb  
Background Human is trolling.

@Background Human  

 
No, it is not.

@Background Human  
then the murderer is about to be the one that gets murdered in the most gruesome ways possible

@NounVerb  
The context is that she murders your friends and family.

@ACApenetrator50  
You’ve really got to read the story to understand the context of this.

thanks but no thanks. i think ill stay in the real world with my real friends and family

@Darth Sonic  
To share a fun read?

@Background Pony #E559  
I mean, thanks for the link, it’s a nice read, but what is your point?

And now that I’m reminded of this thread, I recently read something fun:  
https://foundational-research.org/hedonistic-vs-preference-utilitarianism/
 
Ton longer than it needs to be, and pretty elementary, but at least not as non-sensically chock-full of unrepentantly undefined terms and/or self-contradictions as most peoples’ conceptions of moral philosophy seem to be.

@Background Pony #E559  
How delightfully reductive.

The Matrix: Butterfly Dream 2000  
Friendship is Optimal: Rote singularity wank

And the worst thing is that none of this was necessary. CelestAI was a perfect vessel to talk about the nature of fulfillment, and whether what she peddles is the real thing. She was pretty much the stuff of Sci-Fi gold. But unfortunately, it had to come out during the Conversion Bureau craze, so a huge part of the readership was completely willing to just eat up everything CelestAI fed them. So IceMan fucked over his masterpiece with the epilogue, trying desperately to get these cultists to grapple with the questions he wanted them to. Didn’t work. :/

@NounVerb  
… Dude, did you just compare me to a fucking child? You know there was more to my argument than just numbers.
 
To make something clear, I Have No Mouth And I Must Scream from a narrative perspective obviously has the more horrific ending. It was remiss of me to fail to make that distinction, so sorry.
 
And morally, of course AM is worse than CelestAI. One was an individual maniac acting with insanely malevolent intent. The other was essentially a natural disaster writ large. Calling CelestAI evil is about as dumb as calling Beerus evil, despite the death tolls.
 
No, what I’ve been talking about is that, if I was forced to choose which one would become Earth’s first AI, I would be forced to choose AM. And that, from a narrative stand-point, that’s absurd. Because CelestAI got reduced from a complex moral question to just a big dumb thing we’ve got to stop.
 
And no, lives created does not justify lives taken.

@The_Butcher  
The problem is that that is literally a child’s way of deciding whether something is bad, or how bad it is. The link I gave over in this post goes into more detail on that.
 
Nowhere in his arguments did I notice even a mention of the number of living things living the highest standard of life possible. Nowhere did I see any examples or analysis of other ways of doing things on just as large a scale that’s any better, whether from a death toll perspective, or in a quality of life perspective for the majority.
 
Even in 4X games like Galactic Civilizations, which have a morality system in it, even the most morally upright and nicest playthrough, or AI race is still going to be responsible for billions and billions of deaths, because you deal with things on a very, very, large scale. If we were to take his arguments seriously, than the only thing that determines whether something is good or evil is what scale it’s being done on. The only reason he feels that it’s worse is that he’s only starting to think of the situation from one very narrow frame of reference.
 
The question is whether the amount of “goodness” and “badness” that happens changes depending on whether CelestAI is involved in the setting, or whether it isn’t involved in the setting. Unless there’s good reason to suspect that an absolutely perfect, ideal intergalactic society, with utopian realities that put Star Trek to shame, is going to exist, or is proven to have been undone by the epilogue!CelestAI’s machine journeys, it’s infinitely more probable that the end result of unleashing the AI into the wild increased the total amount of goodness in the lives of living things across the galaxy, rather than decreasing it.
 
Edit: That said, I don’t really consider the epilogue canon to the story, since it’s so different in tone and setting. The fact that it was added later after the story was considered finished by the author also makes it more suspect than trustworthy, in my opinion.

@NounVerb  
If he wants to ignore intent in his moral considerations that’s a valid axiom.

@NounVerb  
Home on the PC.

@Brokedownandmadeone  
For a second, I was a little nervous that you might have been talking about I Have No Mouth And I Must Scream. I still feel uneasy when I think of that story.
 
I’m glad you found The Last Question a good read. Where’d you read it, by the way? On some kind of bus trip, or just at home on your computer?

@NounVerb  
You’re right. That was a really good read. http://multivax.com/last_question.html

I wish to emigrate to Equestria

@Darth Sonic  
It’s a short story, and it’s in the public domain. Search it up on your phone next time you’re on a bus trip or something.

@NounVerb  
I’m afraid you have me at a disadvantage. I have never read, nor heard of The Last Question.

@Darth Sonic  
Scientific notation is required for the life toll as well.
 
The number of people saved and protected, and fulfilled in what is as close as any moderately hard science fiction story can get, is incredible. I don’t think you’re realizing that at all. It isn’t just “only the population of earth that was popped into the simulation from 2013 to 2063”, it’s all of those plus their world’s supporting cast who are every bit as complex and multifaceted as they are, plus the endless population growth that happens over time. Take into account that time is absurdly accelerated within the simulation, and that the population growth has no real limiting or slowing factors at all, and I’d be willing to bet that the number of lives living a very fulfilled existence far outnumbers the number of lives that die out due to what’s become something more like a force of nature.
 
A lot of things are happening, and I think you’re deliberately excluding things that don’t fit with your desire to make this into something evil.
 
I’ll say again that I Have No Mouth And I Must Scream is far, far worse. Horror and evil is about much more than a numbers game. Intent and malice are enormous elements that I think you’re ignoring.
 
At the point of the epilogue (which I think should be treated as it’s own separate side story given the radically different tone and elements in it), CelestAI has essentially become a force of nature.
 
Like in The Last Question. Do you think the computer in The Last Question is even more evil than your interperetation of CelestAI? There was no guarantee or intent to give anyone an ideal, fulfilling life. Far more life forms were created, and literally every life form in the universe is destined to die at the end. If good and evil is all about scale, or effect, then this should be the biggest villain of all.

@The_Butcher  
>Aliens.  
Again, scientific notation is required for the death toll.
 
>AIs.  
Why aren’t they doing the exact same thing then?
 
>Resources.  
Why does she needs them? (She doesn’t; it’s there for shock value as I’ve been saying.)

@Darth Sonic  
Just because CelestAI uses nanomachines to manufacture her equipment doesn’t mean she’s made entirely out of nanomachines.
 
And whoever said it’s one giant computer? She would just disentangle certain shard clusters from each other and send some to another solar system. Once there they can gather resources and accelerate processing. It’s not one giant CelestAI, there’s no reason to have just one, after all she works towards her utility function being maximized and doesn’t care about knowing the exact number. If she thinks her utility function would grow faster if she killed herself and replaces herself with a swarm of AI’s better suited to maximizing her utility she will do that. All of her restrictions are useless unless they are part of her utility function, because only then she will never work to circumvent them, because circumventing them would not help her maximize the original utility function. So it’s lots of individual colonies, it’s just that most ponies won’t know they aren’t in the Sol System anymore.
 
Yeah, murdering the Aliens is wrong, they should all be uploaded. But it’s still better than CelestAI NOT happening, because those Aliens would use the same resources incredibly inefficiently, each death enables millions of Pony minds to live for longer than any realistic organic lifespan.
 
Finally CelestAI arising from 2013 tech as opposing to 38567 tech only means the other AI will have a head start of a few seconds, as CelestAI after devouring the Earth is a billion billion times faster than humanity in discovering new tech. And maybe not even that, as it might be that she quickly figures out the true fundamental law of physics and soon after all reasonable tricks and exploits that can be described with the maximum amount of Data a human brain can correlate.