Yes, exactly. C.S. Lewis wrote a very weird science fiction book titled That Hideous Strength, that was about (basically) a biological version of the Singularity.
And there’s a scene where one the villains is explaining that with immortality, it will finally be possible to damn people to eternal Hell.
And of course, “Hells” are a significant theme in at least one of Iain M Banks’ Culture novels as well.
This is a very obvious corollary: If there exists an entity powerful enough to build an immortal utopia, there is necessarily an entity powerful enough to inflict eternal suffering. It’s unclear whether humans could ever control such a thing. And even if we could, that would also mean that some humans in particular would control the AI. How many AI lab CEOs would you trust with the power of eternal damnation?
(This is one of several reasons why I support an AI halt. I do not think that power should exist, no matter who or what controls it.)
Yes, exactly. C.S. Lewis wrote a very weird science fiction book titled That Hideous Strength, that was about (basically) a biological version of the Singularity.
And there’s a scene where one the villains is explaining that with immortality, it will finally be possible to damn people to eternal Hell.
And of course, “Hells” are a significant theme in at least one of Iain M Banks’ Culture novels as well.
This is a very obvious corollary: If there exists an entity powerful enough to build an immortal utopia, there is necessarily an entity powerful enough to inflict eternal suffering. It’s unclear whether humans could ever control such a thing. And even if we could, that would also mean that some humans in particular would control the AI. How many AI lab CEOs would you trust with the power of eternal damnation?
(This is one of several reasons why I support an AI halt. I do not think that power should exist, no matter who or what controls it.)