Hi, everyone. I’m not sure if my post is well-written, but I think LW might be the only right place to have this discussion. Feel free to suggest changes.
AGI may arrive soon and it is possible that it would kill us all. This does not bother me that much, as dying happens to people all the time. But losing control of our mind and lifespan is a much less pleasant perspective.
As far as I am aware, people usually do not let others kill themselves. In some situations, this may lead to involuntary placement in a mental hospital. And as not living forever may be seen as a form of killing yourself, AGI may quite well not let you have a finite lifespan. That places you in the uncomfortable situation of being trapped with AGI forever.
Going further. We do not know what peak happiness looks like, and our regular state may be very different from it. And as EY outlined in Three Worlds Collide, letting us live our miserable lives may be unacceptable. This change may not align well with our understanding of life’s purpose. And being trapped to live that way forever might not be that desirable.
So my question is: would you see that as a possible outcome? If so, do you see a way to somehow see that happening in advance?
[Question] How likely is AGI to force us all to be happy forever? (much like in the Three Worlds Collide novel)
Hi, everyone. I’m not sure if my post is well-written, but I think LW might be the only right place to have this discussion. Feel free to suggest changes.
AGI may arrive soon and it is possible that it would kill us all. This does not bother me that much, as dying happens to people all the time.
But losing control of our mind and lifespan is a much less pleasant perspective.
As far as I am aware, people usually do not let others kill themselves. In some situations, this may lead to involuntary placement in a mental hospital. And as not living forever may be seen as a form of killing yourself, AGI may quite well not let you have a finite lifespan. That places you in the uncomfortable situation of being trapped with AGI forever.
Going further. We do not know what peak happiness looks like, and our regular state may be very different from it. And as EY outlined in Three Worlds Collide, letting us live our miserable lives may be unacceptable. This change may not align well with our understanding of life’s purpose. And being trapped to live that way forever might not be that desirable.
So my question is: would you see that as a possible outcome? If so, do you see a way to somehow see that happening in advance?