One final thing is that I typically didn’t emphasize loss of control//superintelligence//recursive self-improvement. I didn’t hide it, but I included it in a longer list of threat models
I’d be very interested to see that longer threat model list!
If memory serves me well, I was informed by Hendrycks’ overview of catastrophic risks. I don’t think it’s a perfect categorization, but I think it does a good job laying out some risks that feel “less speculative” (e.g., malicious use, race dynamics as a risk factor that could cause all sorts of threats) while including those that have been painted as “more speculative” (e.g., rogue AIs).
I’ve updated toward the importance of explaining & emphasizing risks from sudden improvements in AI capabilities, AIs that can automate AI research, and intelligence explosions. I also think there’s more appetite for that now than there used to be.
I’d be very interested to see that longer threat model list!
If memory serves me well, I was informed by Hendrycks’ overview of catastrophic risks. I don’t think it’s a perfect categorization, but I think it does a good job laying out some risks that feel “less speculative” (e.g., malicious use, race dynamics as a risk factor that could cause all sorts of threats) while including those that have been painted as “more speculative” (e.g., rogue AIs).
I’ve updated toward the importance of explaining & emphasizing risks from sudden improvements in AI capabilities, AIs that can automate AI research, and intelligence explosions. I also think there’s more appetite for that now than there used to be.