There’s also a good scenario where the US develops an AGI that is capable of slowing down rival AGI development, but not so capable and misaligned that it causes serious problems, and that gives people enough time to solve alignment enough to bootstrap to AI solving alignment.
I’m feeling somewhat optimistic about this, because the workload involved in slowing down a rival AGI development doesn’t seem so high that it couldn’t be monitored/understood fully or mostly by humans, and the capabilities required also doesn’t seem so high that any AI that could do it would be inherently very dangerous or hard to control.
You’ve probably thought more about this scenario than I have, so I’d be interested in hearing more about how you think it will play out. (Do you have links to where you’ve discussed it previously?) I was speaking mostly in relative terms, as slowing down rival AGI efforts in the ways I described seems more promising/realistic/safer than any other “pivotal acts” I had previously heard or thought of.
My overall sense is that with substantial commited effort (but no need for fundamental advances) and some amount of within US coordination, it’s reasonably, but not amazingly, likely to work. (See here for some discussion.)
I think the likelihood of well executed substantial commited effort isn’t that high though, maybe 50%. And sufficient within US coordination also seems unclear.
There’s also a good scenario where the US develops an AGI that is capable of slowing down rival AGI development, but not so capable and misaligned that it causes serious problems, and that gives people enough time to solve alignment enough to bootstrap to AI solving alignment.
I’m feeling somewhat optimistic about this, because the workload involved in slowing down a rival AGI development doesn’t seem so high that it couldn’t be monitored/understood fully or mostly by humans, and the capabilities required also doesn’t seem so high that any AI that could do it would be inherently very dangerous or hard to control.
I think I disagree with your optimism, but I don’t feel confident. I agree that things could work out as you hope.
You’ve probably thought more about this scenario than I have, so I’d be interested in hearing more about how you think it will play out. (Do you have links to where you’ve discussed it previously?) I was speaking mostly in relative terms, as slowing down rival AGI efforts in the ways I described seems more promising/realistic/safer than any other “pivotal acts” I had previously heard or thought of.
My overall sense is that with substantial commited effort (but no need for fundamental advances) and some amount of within US coordination, it’s reasonably, but not amazingly, likely to work. (See here for some discussion.)
I think the likelihood of well executed substantial commited effort isn’t that high though, maybe 50%. And sufficient within US coordination also seems unclear.