Disaster doesn’t need to make humans extinct to stop AI takeover. Only destroy enough of our excess technological capacity to make it impossible for a hostile AI to come into existence with enough infrastructure to bootstrap itself to self-sufficiency where it doesn’t need humans anymore.
I guess it’s possible that the human population could drop by 90%, lose most tech and manufacturing capabilities, and recover within a few generations and pick up where today’s AI left off, only delaying the takeover. Kind of goes to the question of “what is doomed”? In some sense, every individual and species including the AI that FOOMs is doomed—something else will eventually replace it.
I don’t personally expect a (total) nuclear war to happen before AGI. But, conditional on total nuclear war before AGI: although nuclear war is unlikely to cause human extinction, it could still cause a big setback in civilization’s technological infrastructure, which would significantly delay AGI.
Why?
Partly asking because of Nuclear War Is Unlikely To Cause Human Extinction (I don’t think the post’s case is ironclad, but it wouldn’t be my mainline belief)
Disaster doesn’t need to make humans extinct to stop AI takeover. Only destroy enough of our excess technological capacity to make it impossible for a hostile AI to come into existence with enough infrastructure to bootstrap itself to self-sufficiency where it doesn’t need humans anymore.
I guess it’s possible that the human population could drop by 90%, lose most tech and manufacturing capabilities, and recover within a few generations and pick up where today’s AI left off, only delaying the takeover. Kind of goes to the question of “what is doomed”? In some sense, every individual and species including the AI that FOOMs is doomed—something else will eventually replace it.
I don’t personally expect a (total) nuclear war to happen before AGI. But, conditional on total nuclear war before AGI: although nuclear war is unlikely to cause human extinction, it could still cause a big setback in civilization’s technological infrastructure, which would significantly delay AGI.