Some people seem to think my timelines have shifted a bunch while they’ve only moderately changed.
Relative to my views at the start of 2025, my median (50th percentile) for AIs fully automating AI R&D was pushed back by around 2 years—from something like Jan 2032 to Jan 2034. My 25th percentile has shifted similarly (though perhaps more importantly) from maybe July 2028 to July 2030. Obviously, my numbers aren’t fully precise and vary some over time. (E.g., I’m not sure I would have quoted these exact numbers for this exact milestone at the start of the year; these numbers for the start of the year are partially reverse engineered from this comment.)
Fully automating AI R&D is a pretty high milestone; my current numbers for something like “AIs accelerate AI R&D as much as what would happen if employees ran 10x faster (e.g. by ~fully automating research engineering and some other tasks)” are probably 50th percentile Jan 2032 and 25th percentile Jan 2029.[1]
I’m partially posting this so there is a record of my views; I think it’s somewhat interesting to observe this over time. (That said, I don’t want to anchor myself, which does seem like a serious downside. I should slide around a bunch and be somewhat incoherent if I’m updating as much as I should: my past views are always going to be somewhat obviously confused from the perspective of my current self.)
What are your very long timeline expectations, for 2045+ or 2055+ AGI (automated AI R&D, sure)? That’s where I expect most of the rare futures with humanity not permanently disempowered to be, though the majority even of these long timelines will still result in permanent disempowerment (or extinction).
I think it takes at least about 10 years to qualitatively transform an active field of technical study or change the social agenda, so 2-3 steps of such change might have a chance of sufficiently reshaping how the world thinks about AI x-risk and what technical tools are available for shaping minds of AGIs, in order to either make a human-initiated lasting Pause plausible, or to have the means of aligning AGIs in an ambitious sense.
I appreciate you saying that the 25th percentile timeline might be more important. I think that’s right and underappreciated.
One of your recent (excellent) posts also made me notice that AGI timelines probably aren’t normally distributed. Breakthroughs, other large turns of events, or large theoretical misunderstandings at this point probably play a large role, and there are probably only a very few of those that will hit. Small unpredictable events that create normal distributions will play a lesser role.
I don’t know how you’d characterize that mathematically, but I don’t think it’s right to assume it’s normally distributed, or even close.
Back to your comment on the 25th percentile being important: I think there’s a common error where people round to the median and then think “ok, that’s probably when we need to have alignment/strategy figured out.” You’d really want to have it at least somewhat ready far earlier.
That’s both in case it’s on the earlier side of the predicted distribution, and because alignment theory and practice need to be ready far enough in advance of game time to have diffused and be implemented for the first takeover-capable model.
I’ve been thinking of writing a post called something like “why are so few people frantic about alignment?” making those points. Stated timeline distributions don’t seem to match mood IMO and I’m trying to figure out why. I realize that part of it is a very reasonable “we’ll figure it out when/if we get there.” And perhaps others share my emotional dissociation from my intellectual expectations. But maybe we should all be a bit more frantic. I’d like some more halfassed alignment solutions in play and under discussion right now. The 80⁄20 rule probably applies here.
I did have similar timelines, but I have generally moved them forward to ~ 2027-2028 after updating my priors with OpenAI/XAI employee X posts. They are generally confident that post-training scaleups will accelerate timelines, as well as stating that post-training can be scaled, with return to match, well beyond that of pre-training. As they have inside information, I am inclined to trust them. Insiders like Anthropic’s Jack Clark are even more bold, he says that superintelligent machines will be available late 2026.
Some people seem to think my timelines have shifted a bunch while they’ve only moderately changed.
Relative to my views at the start of 2025, my median (50th percentile) for AIs fully automating AI R&D was pushed back by around 2 years—from something like Jan 2032 to Jan 2034. My 25th percentile has shifted similarly (though perhaps more importantly) from maybe July 2028 to July 2030. Obviously, my numbers aren’t fully precise and vary some over time. (E.g., I’m not sure I would have quoted these exact numbers for this exact milestone at the start of the year; these numbers for the start of the year are partially reverse engineered from this comment.)
Fully automating AI R&D is a pretty high milestone; my current numbers for something like “AIs accelerate AI R&D as much as what would happen if employees ran 10x faster (e.g. by ~fully automating research engineering and some other tasks)” are probably 50th percentile Jan 2032 and 25th percentile Jan 2029.[1]
I’m partially posting this so there is a record of my views; I think it’s somewhat interesting to observe this over time. (That said, I don’t want to anchor myself, which does seem like a serious downside. I should slide around a bunch and be somewhat incoherent if I’m updating as much as I should: my past views are always going to be somewhat obviously confused from the perspective of my current self.)
While I’m giving these numbers, note that I think Precise AGI timelines don’t matter that much.
See this comment for the numbers I would have given for this milestone at the start of the year.
What are your very long timeline expectations, for 2045+ or 2055+ AGI (automated AI R&D, sure)? That’s where I expect most of the rare futures with humanity not permanently disempowered to be, though the majority even of these long timelines will still result in permanent disempowerment (or extinction).
I think it takes at least about 10 years to qualitatively transform an active field of technical study or change the social agenda, so 2-3 steps of such change might have a chance of sufficiently reshaping how the world thinks about AI x-risk and what technical tools are available for shaping minds of AGIs, in order to either make a human-initiated lasting Pause plausible, or to have the means of aligning AGIs in an ambitious sense.
I appreciate you saying that the 25th percentile timeline might be more important. I think that’s right and underappreciated.
One of your recent (excellent) posts also made me notice that AGI timelines probably aren’t normally distributed. Breakthroughs, other large turns of events, or large theoretical misunderstandings at this point probably play a large role, and there are probably only a very few of those that will hit. Small unpredictable events that create normal distributions will play a lesser role.
I don’t know how you’d characterize that mathematically, but I don’t think it’s right to assume it’s normally distributed, or even close.
Back to your comment on the 25th percentile being important: I think there’s a common error where people round to the median and then think “ok, that’s probably when we need to have alignment/strategy figured out.” You’d really want to have it at least somewhat ready far earlier.
That’s both in case it’s on the earlier side of the predicted distribution, and because alignment theory and practice need to be ready far enough in advance of game time to have diffused and be implemented for the first takeover-capable model.
I’ve been thinking of writing a post called something like “why are so few people frantic about alignment?” making those points. Stated timeline distributions don’t seem to match mood IMO and I’m trying to figure out why. I realize that part of it is a very reasonable “we’ll figure it out when/if we get there.” And perhaps others share my emotional dissociation from my intellectual expectations. But maybe we should all be a bit more frantic. I’d like some more halfassed alignment solutions in play and under discussion right now. The 80⁄20 rule probably applies here.
I think it would be interesting if you replied to this comment once a year or so to report how your timelines have changed.
comment on a year old post may not be the best place, maybe a new short form on this day yearly which links to all previous posts?
I did have similar timelines, but I have generally moved them forward to ~ 2027-2028 after updating my priors with OpenAI/XAI employee X posts. They are generally confident that post-training scaleups will accelerate timelines, as well as stating that post-training can be scaled, with return to match, well beyond that of pre-training. As they have inside information, I am inclined to trust them. Insiders like Anthropic’s Jack Clark are even more bold, he says that superintelligent machines will be available late 2026.