I don’t believe that “current AI is at human intelligence in most areas”. I think that it is superhuman in a few areas, within the human range in some areas, and subhuman in many areas—especially areas where the things you’re trying to do are not well specified tasks.
I’m not sure how to weight people who think most about how to build AGI vs more general AI researchers (median says HLAI in 2059, p(Doom) 5-10%) vs forecasters more generally. There’s a difference in how much people have thought about it, but also selection bias: most people who are skeptical of AGI soon are likely not going to work in alignment circles or an AGI lab. The relevant reference class is not the Wright Brothers, since hindsight tells us that they were the ones who succeeded. One relevant reference class is the Society for the Encouragement of Aerial Locomotion by means of Heavier-than-Air Machines, founded in 1863, although I don’t know what their predictions were. It might also make sense to include many groups of futurists focusing on many potential technologies, rather than just on one technology that we know worked out.
I agree that there’s a heavy self-selection bias for those working in safety or AGI labs. So I’d say both of these factors are large, and how to balance them is unclear.
I agree that you can’t use the Wright Brothers as a reference class, because you don’t know in advance who’s going to succeed.
I do want to draw a distinction between AI researchers, who think about improving narrow ML systems, and AGI researchers. There are people who spend much more time thinking about how breakthroughs to next-level abilities might be achieved, and what a fully agentic, human-level AGI would be like. The line is fuzzy, but I’d say these two ends of a spectrum exist. I’d say the AGI researchers are more like the society for aerial locomotion. I assume that society had a much better prediction than the class of engineers who’d rarely thought about integrating their favorite technologies (sailmaking, bicycle design, internal combustion engine design) into flying machines.
I don’t believe that “current AI is at human intelligence in most areas”. I think that it is superhuman in a few areas, within the human range in some areas, and subhuman in many areas—especially areas where the things you’re trying to do are not well specified tasks.
I’m not sure how to weight people who think most about how to build AGI vs more general AI researchers (median says HLAI in 2059, p(Doom) 5-10%) vs forecasters more generally. There’s a difference in how much people have thought about it, but also selection bias: most people who are skeptical of AGI soon are likely not going to work in alignment circles or an AGI lab. The relevant reference class is not the Wright Brothers, since hindsight tells us that they were the ones who succeeded. One relevant reference class is the Society for the Encouragement of Aerial Locomotion by means of Heavier-than-Air Machines, founded in 1863, although I don’t know what their predictions were. It might also make sense to include many groups of futurists focusing on many potential technologies, rather than just on one technology that we know worked out.
I agree that there’s a heavy self-selection bias for those working in safety or AGI labs. So I’d say both of these factors are large, and how to balance them is unclear.
I agree that you can’t use the Wright Brothers as a reference class, because you don’t know in advance who’s going to succeed.
I do want to draw a distinction between AI researchers, who think about improving narrow ML systems, and AGI researchers. There are people who spend much more time thinking about how breakthroughs to next-level abilities might be achieved, and what a fully agentic, human-level AGI would be like. The line is fuzzy, but I’d say these two ends of a spectrum exist. I’d say the AGI researchers are more like the society for aerial locomotion. I assume that society had a much better prediction than the class of engineers who’d rarely thought about integrating their favorite technologies (sailmaking, bicycle design, internal combustion engine design) into flying machines.