I think they’re way off. I was visualizing my self at Christmastime 2027, sipping eggnog and gloating about how right I was,
Reading further it seems like you are basically just saying “Timelines are longer than 2027.” You’ll be interested to know that we actually all agree on that. Perhaps you are more confident than us; what are your timelines exactly? Where is your 50% mark for the superhuman coder milestone being reached? (Or if you prefer a different milestone like AGI or ASI, go ahead and say that)
Unfortunately, it’s hard to predict it. I did describe how Grok 4[1] and GPT-5 are arguably evidence that the accelerated doubling trend between GPT4o and o3 is replaced by something slower. As far as I understand, were the slower trend to repeat METR’s original law (GPT2-GPT4?[2]), we would obtain the 2030s.
But, as you remark, “we should have some credence on new breakthroughs<...> that would lead to superhuman coders within a year or two, after being appropriately scaled up and tinkered with.” The actual probability of the breakthrough is likely a crux: you believe it to be 8% a year and I think of potential architectures waiting to be tried. One such architecture is diffusion models[3] which have actually been previewed and could be waiting to be released.
So assuming world peace, the timeline could end up being modeled by a combination of scaling compute up and few algorithmic breakthroughs with random acceleration effects, and each breakthrough would have to be somehow distributed by the amount of research done, then have the most powerful Agent trained to use the breakthrough, as happens with Agent-3 and Agent-4 created from Agent-2 in the forecast.
The worse-case scenario[4] also has timelines affected by compute deficiency. For instance, the Taiwan invasion is thought to happen by 2027 and could be likely to prompt the USG to force the companies to merge and to race (to AI takeover) as hard as they can.
GPT-4 and GPT-4o were released in March 2023 and May 2024 and had only one doubling in 14 months. Something hit a plateau, then in June 2024 Anthropic released Claude 3.5 Sonnet (old), and a new trend began. As of now, the trend likely ended at o3, and Grok 4 and GPT5 are apparently in the same paradigm which could have faced efficiency limits.
Hey Daniel, I loved the podcast with Dwarkesh and Scott Alexander. I am glad you have gotten people talking about this, though I’m of two minds about it, because as I say in my post, I believe your estimates in the AI 2027 document are very aggressive (and there were some communication issues there I discussed with Eli in another comment). I worry what might happen in 2028 if basically the entire scenario described on the main page turns out to not happen, which is what I believe.
My blog post is a reaction to the AI 2027 document as it stands, which doesn’t seem to have any banner at the top saying the authors no longer endorse the findings or anything. The domain name and favicon haven’t changed. I am now aware that you have adjusted your median timeline from 2027 to 2028, and that Eli has adjusted his estimate as well. The scenario itself describes some pretty crazy things happening in 2027, and the median estimate in the research (before adjustments) seems to be 2027 for SC, SAR, SAIR, and ASI all in 2027. I definitely don’t agree that any of those milestones will be reached in 2027, nor in 2028.
Of course, predicting the future is hard the further out in time we go. I have a strongish sense for 2027, but I would put SC, SAR, SAIR, and ASI all at least ten years further than that, and it’s really hard to know what the heck will be going on in the world by then, because unlike you I don’t believe this is a matter of continuous progress (for reasons mentioned in my post), and discontinuous progress is hard to predict. In 1902 Simon Newcomb said, “Flight by machines heavier than air is unpractical and insignificant, if not utterly impossible.” The next year the Wright brothers took off. I’m not sure trying to put a number on it is better than simply saying, “I don’t know.”
In the spirit of answering the question you asked, I’d predict SC in the year 2050. But this is such a low-confidence prediction as to be essentially worthless, like saying I think there’s a 50% God exists.
I’d like to know what you thought of my justification for my doubts about LLMs, if you had time to get that far.
Thanks for the critique & the reply btw! Very much appreciate you giving quantitative alternative credences to mine, it’s a productive way to focus the conversation I think.
which doesn’t seem to have any banner at the top saying the authors no longer endorse the findings or anything. The domain name and favicon haven’t changed. I am now aware that you have adjusted your median timeline from 2027 to 2028, and that Eli has adjusted his estimate as well.
Footnote #1 used to be attached to “We wrote a scenario...” but we wanted it to be more prominent in response to exactly the sort of criticism you are making, so we moved it up to be literally a footnote on the title. I suppose we could have put it in the main text itself.
My median is currently 2029 actually; at the time AI 2027 was published it was 2028.
I have a strongish sense for 2027, but I would put SC, SAR, SAIR, and ASI all at least ten years further than that, and it’s really hard to know what the heck will be going on in the world by then, because unlike you I don’t believe this is a matter of continuous progress (for reasons mentioned in my post), and discontinuous progress is hard to predict. In 1902 Simon Newcomb said, “Flight by machines heavier than air is unpractical and insignificant, if not utterly impossible.” The next year the Wright brothers took off. I’m not sure trying to put a number on it is better than simply saying, “I don’t know.”
OK, thanks, that’s helpful. So yeah, while we agree that SC probably won’t happen by 2027 EOY, we do still have a disagreement—I think it probably WILL happen in the next five years or so (and the rest of the team thinks it’ll probably happen in the next ten years or so) whereas you seem confident it WON’T happen before 2037. I hope you are right! I agree also that the future is very hard to predict, especially the farther out it is (and 2037+ is very far out)
There’s a lot to say about why I think SC will probably happen in the next five years or so. I’ll go leave line-by-line comments in the relevant section of your post!
Yeah, someone pointed out that footnote to me, and I laughed a bit. It’s very small and easy to miss. I don’t think you guys actually misrepresented anything. It’s clear from reading the research section what your actual timelines are and so on. I’m just pointing to communication issues.
Talking about 2027, the authors did inform the readers in a footnote, but revisions of the timelines forecast turned out to be hard to deliver to the general public. Let’s wait for @Daniel Kokotajlo to state his opinion on the doubts related to SOTA architecture. In my opinion these problems would be resolved by a neuralese architecture or an architecture which could be an even bigger breakthrough (neuralese with big internal memory?)
Reading further it seems like you are basically just saying “Timelines are longer than 2027.” You’ll be interested to know that we actually all agree on that. Perhaps you are more confident than us; what are your timelines exactly? Where is your 50% mark for the superhuman coder milestone being reached? (Or if you prefer a different milestone like AGI or ASI, go ahead and say that)
Unfortunately, it’s hard to predict it. I did describe how Grok 4[1] and GPT-5 are arguably evidence that the accelerated doubling trend between GPT4o and o3 is replaced by something slower. As far as I understand, were the slower trend to repeat METR’s original law (GPT2-GPT4?[2]), we would obtain the 2030s.
But, as you remark, “we should have some credence on new breakthroughs<...> that would lead to superhuman coders within a year or two, after being appropriately scaled up and tinkered with.” The actual probability of the breakthrough is likely a crux: you believe it to be 8% a year and I think of potential architectures waiting to be tried. One such architecture is diffusion models[3] which have actually been previewed and could be waiting to be released.
So assuming world peace, the timeline could end up being modeled by a combination of scaling compute up and few algorithmic breakthroughs with random acceleration effects, and each breakthrough would have to be somehow distributed by the amount of research done, then have the most powerful Agent trained to use the breakthrough, as happens with Agent-3 and Agent-4 created from Agent-2 in the forecast.
Maybe a blog post explaining more about your timelines and how they’ve updated would help?
The worse-case scenario[4] also has timelines affected by compute deficiency. For instance, the Taiwan invasion is thought to happen by 2027 and could be likely to prompt the USG to force the companies to merge and to race (to AI takeover) as hard as they can.
Grok 4 is also known to have been trained by spending similar amounts of compute on pretraining and RL. Is it also known about GPT-5?
GPT-4 and GPT-4o were released in March 2023 and May 2024 and had only one doubling in 14 months. Something hit a plateau, then in June 2024 Anthropic released Claude 3.5 Sonnet (old), and a new trend began. As of now, the trend likely ended at o3, and Grok 4 and GPT5 are apparently in the same paradigm which could have faced efficiency limits.
They do rapidly generate text (e.g. code). But I don’t understand how they, say, decide to look important facts up.
Of course, the absolute worst scenario is a black swan like currency collapse or China’s responce with missile strikes.
Yeah we are working on it sorry!
Hey Daniel, I loved the podcast with Dwarkesh and Scott Alexander. I am glad you have gotten people talking about this, though I’m of two minds about it, because as I say in my post, I believe your estimates in the AI 2027 document are very aggressive (and there were some communication issues there I discussed with Eli in another comment). I worry what might happen in 2028 if basically the entire scenario described on the main page turns out to not happen, which is what I believe.
My blog post is a reaction to the AI 2027 document as it stands, which doesn’t seem to have any banner at the top saying the authors no longer endorse the findings or anything. The domain name and favicon haven’t changed. I am now aware that you have adjusted your median timeline from 2027 to 2028, and that Eli has adjusted his estimate as well. The scenario itself describes some pretty crazy things happening in 2027, and the median estimate in the research (before adjustments) seems to be 2027 for SC, SAR, SAIR, and ASI all in 2027. I definitely don’t agree that any of those milestones will be reached in 2027, nor in 2028.
Of course, predicting the future is hard the further out in time we go. I have a strongish sense for 2027, but I would put SC, SAR, SAIR, and ASI all at least ten years further than that, and it’s really hard to know what the heck will be going on in the world by then, because unlike you I don’t believe this is a matter of continuous progress (for reasons mentioned in my post), and discontinuous progress is hard to predict. In 1902 Simon Newcomb said, “Flight by machines heavier than air is unpractical and insignificant, if not utterly impossible.” The next year the Wright brothers took off. I’m not sure trying to put a number on it is better than simply saying, “I don’t know.”
In the spirit of answering the question you asked, I’d predict SC in the year 2050. But this is such a low-confidence prediction as to be essentially worthless, like saying I think there’s a 50% God exists.
I’d like to know what you thought of my justification for my doubts about LLMs, if you had time to get that far.
Thanks for the critique & the reply btw! Very much appreciate you giving quantitative alternative credences to mine, it’s a productive way to focus the conversation I think.
Footnote #1 used to be attached to “We wrote a scenario...” but we wanted it to be more prominent in response to exactly the sort of criticism you are making, so we moved it up to be literally a footnote on the title. I suppose we could have put it in the main text itself.
My median is currently 2029 actually; at the time AI 2027 was published it was 2028.
OK, thanks, that’s helpful. So yeah, while we agree that SC probably won’t happen by 2027 EOY, we do still have a disagreement—I think it probably WILL happen in the next five years or so (and the rest of the team thinks it’ll probably happen in the next ten years or so) whereas you seem confident it WON’T happen before 2037. I hope you are right! I agree also that the future is very hard to predict, especially the farther out it is (and 2037+ is very far out)
There’s a lot to say about why I think SC will probably happen in the next five years or so. I’ll go leave line-by-line comments in the relevant section of your post!
Yeah, someone pointed out that footnote to me, and I laughed a bit. It’s very small and easy to miss. I don’t think you guys actually misrepresented anything. It’s clear from reading the research section what your actual timelines are and so on. I’m just pointing to communication issues.
Thanks for your responses! I’ll check them out.
Talking about 2027, the authors did inform the readers in a footnote, but revisions of the timelines forecast turned out to be hard to deliver to the general public. Let’s wait for @Daniel Kokotajlo to state his opinion on the doubts related to SOTA architecture. In my opinion these problems would be resolved by a neuralese architecture or an architecture which could be an even bigger breakthrough (neuralese with big internal memory?)