A lot of people seem to be shrugging this question off, saying basically, “Transhuman minds are ineffable, we can’t imagine what they would do.” If we have some kind of AI god that rapidly takes over the world after a hard takeoff, then I think that logic basically applies. The world after that point will reflect the values implemented into the AI god.
Robin Hanson has described a different scenario, that I take somewhat more seriously than the AI god scenario.
This long competition has not selected a few idle gods using vast powers to indulge arbitrary whims, or solar system-sized Matrioshka Brains. Instead, frontier life is as hard and tough as the lives of most wild plants and animals nowadays, and of most subsistence-level humans through history. These hard-life descendants, whom we will call
“colonists,” can still find value, nobility, and happiness in their lives. For them, their tough competitive life is just the “way things are” and have long been.
Hanson is describing a return to the Malthusian condition that has defined life since the beginning. The assumptions seem fairly strong to me:
Competition won’t cease.
The darwinian drive to reproduce will remain (because it is adaptive).
Resources are functionally limited, there is only so much usable energy.
Life will be hard, there will be lots of people “starving” (e.g. not being able to access enough energy to keep functioning, or not being able to buy replacement parts/server space) at the margins, as it is in the natural environment.
A lot of people seem to be shrugging this question off, saying basically, “Transhuman minds are ineffable, we can’t imagine what they would do.” If we have some kind of AI god that rapidly takes over the world after a hard takeoff, then I think that logic basically applies. The world after that point will reflect the values implemented into the AI god.
Robin Hanson has described a different scenario, that I take somewhat more seriously than the AI god scenario.
Hanson is describing a return to the Malthusian condition that has defined life since the beginning. The assumptions seem fairly strong to me:
Competition won’t cease.
The darwinian drive to reproduce will remain (because it is adaptive).
Resources are functionally limited, there is only so much usable energy.
Life will be hard, there will be lots of people “starving” (e.g. not being able to access enough energy to keep functioning, or not being able to buy replacement parts/server space) at the margins, as it is in the natural environment.