Given superintelligence, what happens next depends on the success of the alignment project. The two options:
It fails, and we die soon thereafter (or worse).
It succeeds, and we now have an entity that can solve problems for us far better than any human or human organization. We are now in a world where humans have zero socioeconomic utility. The ASI can create entertainment and comfort that surpasses anything any human can provide. Sure you can still interact with others willing to interact with you, it just won’t be as fun as whatever stimulus the ASI can provide, and both your pool of available playmates and your own willingness to partake will shrink as the ASI gets better at artificially generating the stimuli and emotions you want. We will spend eternity in this state thanks to advanced medicine. Unless the ASI recognizes a right to die, not that many would choose invoke it given the infinite bliss.
Am I missing something? No matter what, it’s beginning to look like the afterlife is fast approaching, whether we die or not. What a life.
I still think a world we don’t see superintelligence in our lifetimes is technically possible, though the chance of that goes down continuously and is already vanishingly small in my view (many experts and pundits disagree). I also think its important not to over-predict regarding what option 2 would look like, there are infinite possibilities and this is only one (eg I could imagine a world where some aligned superintelligence steers us away from infinite dopamine simulation and into a idealized version of the world we live in now, think the Scythe novel series. On the bad side I could imagine a world where superintelligence is controlled by one malevolent entity and we live in a “mid” or even dystopic society for no other reason than to satisfy the class that retains control).
However, yes I agree. We probably live in the most consequential time in all of history, which is exciting, humbling, and scary. Don’t let it get to your head and don’t lose yourself in thoughts of the future lest you forget the beauty of the present. Do your best to help if you can!
Given superintelligence, what happens next depends on the success of the alignment project. The two options:
It fails, and we die soon thereafter (or worse).
It succeeds, and we now have an entity that can solve problems for us far better than any human or human organization. We are now in a world where humans have zero socioeconomic utility. The ASI can create entertainment and comfort that surpasses anything any human can provide. Sure you can still interact with others willing to interact with you, it just won’t be as fun as whatever stimulus the ASI can provide, and both your pool of available playmates and your own willingness to partake will shrink as the ASI gets better at artificially generating the stimuli and emotions you want. We will spend eternity in this state thanks to advanced medicine. Unless the ASI recognizes a right to die, not that many would choose invoke it given the infinite bliss.
Am I missing something? No matter what, it’s beginning to look like the afterlife is fast approaching, whether we die or not. What a life.
I still think a world we don’t see superintelligence in our lifetimes is technically possible, though the chance of that goes down continuously and is already vanishingly small in my view (many experts and pundits disagree). I also think its important not to over-predict regarding what option 2 would look like, there are infinite possibilities and this is only one (eg I could imagine a world where some aligned superintelligence steers us away from infinite dopamine simulation and into a idealized version of the world we live in now, think the Scythe novel series. On the bad side I could imagine a world where superintelligence is controlled by one malevolent entity and we live in a “mid” or even dystopic society for no other reason than to satisfy the class that retains control).
However, yes I agree. We probably live in the most consequential time in all of history, which is exciting, humbling, and scary. Don’t let it get to your head and don’t lose yourself in thoughts of the future lest you forget the beauty of the present. Do your best to help if you can!