Probability of messing it up in some other way during a period of accelerated technological change (e.g. driving ourselves crazy, creating a permanent dystopia, making unwise commitments…): 15%
Do you think these risks can also be reduced by 10x by a “very good RSP”? If yes, how or by what kinds of policies? If not, isn’t “cut risk dramatically [...] perhaps a 10x reduction” kind of misleading?
It concerns me that none of the RSP documents or discussions I’ve seen talked about these particular risks, or “unknown unknowns” (other risks that we haven’t thought of yet).
I’m also bummed that “AI pause” people don’t talk about these risks either, but at least an AI pause would implicitly address these risks by default, whereas RSPs would not.
I don’t think an RSP will be able to address these risks, and I think very few AI policies would address these risks either. An AI pause could address them primarily by significantly slowing human technological development, and if that happened today I’m not even really these risks are getting better at an appreciable rate (if the biggest impact is the very slow thinking from a very small group of people who care about them, then I think that’s a very small impact). I think that in that regime random political and social consequences of faster or slower technological development likely dominate the direct effects from becoming better prepared over time. I would have the same view in retrospect about e.g. a possible pause on AI development 6 years ago. I think at that point the amount of quality-adjusted work on alignment was probably higher than the quality-adjusted work on these kinds of risks today, but still the direct effects on increasingly alignment preparedness would be pretty tiny compared to random other incidental effects of a pause on the AI landscape.
if the biggest impact is the very slow thinking from a very small group of people who care about them, then I think that’s a very small impact
I guess from my perspective, the biggest impact is the possibility that the idea of better preparing for these risks becomes a lot more popular. An analogy with Bitcoin comes to mind, where the idea of cryptography-based distributed money languished for many years, known only to a tiny community, and then was suddenly everywhere. An AI pause would provide more time for something like that to happen. And if the idea of better preparing for these risks was actually a good one (as you seem to think), there’s no reason why it couldn’t (or was very unlikely to) spread beyond a very small group, do you agree?
In My views on “doom” you wrote:
Do you think these risks can also be reduced by 10x by a “very good RSP”? If yes, how or by what kinds of policies? If not, isn’t “cut risk dramatically [...] perhaps a 10x reduction” kind of misleading?
It concerns me that none of the RSP documents or discussions I’ve seen talked about these particular risks, or “unknown unknowns” (other risks that we haven’t thought of yet).
I’m also bummed that “AI pause” people don’t talk about these risks either, but at least an AI pause would implicitly address these risks by default, whereas RSPs would not.
I don’t think an RSP will be able to address these risks, and I think very few AI policies would address these risks either. An AI pause could address them primarily by significantly slowing human technological development, and if that happened today I’m not even really these risks are getting better at an appreciable rate (if the biggest impact is the very slow thinking from a very small group of people who care about them, then I think that’s a very small impact). I think that in that regime random political and social consequences of faster or slower technological development likely dominate the direct effects from becoming better prepared over time. I would have the same view in retrospect about e.g. a possible pause on AI development 6 years ago. I think at that point the amount of quality-adjusted work on alignment was probably higher than the quality-adjusted work on these kinds of risks today, but still the direct effects on increasingly alignment preparedness would be pretty tiny compared to random other incidental effects of a pause on the AI landscape.
I guess from my perspective, the biggest impact is the possibility that the idea of better preparing for these risks becomes a lot more popular. An analogy with Bitcoin comes to mind, where the idea of cryptography-based distributed money languished for many years, known only to a tiny community, and then was suddenly everywhere. An AI pause would provide more time for something like that to happen. And if the idea of better preparing for these risks was actually a good one (as you seem to think), there’s no reason why it couldn’t (or was very unlikely to) spread beyond a very small group, do you agree?