Hey Daniel, I loved the podcast with Dwarkesh and Scott Alexander. I am glad you have gotten people talking about this, though I’m of two minds about it, because as I say in my post, I believe your estimates in the AI 2027 document are very aggressive (and there were some communication issues there I discussed with Eli in another comment). I worry what might happen in 2028 if basically the entire scenario described on the main page turns out to not happen, which is what I believe.
My blog post is a reaction to the AI 2027 document as it stands, which doesn’t seem to have any banner at the top saying the authors no longer endorse the findings or anything. The domain name and favicon haven’t changed. I am now aware that you have adjusted your median timeline from 2027 to 2028, and that Eli has adjusted his estimate as well. The scenario itself describes some pretty crazy things happening in 2027, and the median estimate in the research (before adjustments) seems to be 2027 for SC, SAR, SAIR, and ASI all in 2027. I definitely don’t agree that any of those milestones will be reached in 2027, nor in 2028.
Of course, predicting the future is hard the further out in time we go. I have a strongish sense for 2027, but I would put SC, SAR, SAIR, and ASI all at least ten years further than that, and it’s really hard to know what the heck will be going on in the world by then, because unlike you I don’t believe this is a matter of continuous progress (for reasons mentioned in my post), and discontinuous progress is hard to predict. In 1902 Simon Newcomb said, “Flight by machines heavier than air is unpractical and insignificant, if not utterly impossible.” The next year the Wright brothers took off. I’m not sure trying to put a number on it is better than simply saying, “I don’t know.”
In the spirit of answering the question you asked, I’d predict SC in the year 2050. But this is such a low-confidence prediction as to be essentially worthless, like saying I think there’s a 50% God exists.
I’d like to know what you thought of my justification for my doubts about LLMs, if you had time to get that far.
Thanks for the critique & the reply btw! Very much appreciate you giving quantitative alternative credences to mine, it’s a productive way to focus the conversation I think.
which doesn’t seem to have any banner at the top saying the authors no longer endorse the findings or anything. The domain name and favicon haven’t changed. I am now aware that you have adjusted your median timeline from 2027 to 2028, and that Eli has adjusted his estimate as well.
Footnote #1 used to be attached to “We wrote a scenario...” but we wanted it to be more prominent in response to exactly the sort of criticism you are making, so we moved it up to be literally a footnote on the title. I suppose we could have put it in the main text itself.
My median is currently 2029 actually; at the time AI 2027 was published it was 2028.
I have a strongish sense for 2027, but I would put SC, SAR, SAIR, and ASI all at least ten years further than that, and it’s really hard to know what the heck will be going on in the world by then, because unlike you I don’t believe this is a matter of continuous progress (for reasons mentioned in my post), and discontinuous progress is hard to predict. In 1902 Simon Newcomb said, “Flight by machines heavier than air is unpractical and insignificant, if not utterly impossible.” The next year the Wright brothers took off. I’m not sure trying to put a number on it is better than simply saying, “I don’t know.”
OK, thanks, that’s helpful. So yeah, while we agree that SC probably won’t happen by 2027 EOY, we do still have a disagreement—I think it probably WILL happen in the next five years or so (and the rest of the team thinks it’ll probably happen in the next ten years or so) whereas you seem confident it WON’T happen before 2037. I hope you are right! I agree also that the future is very hard to predict, especially the farther out it is (and 2037+ is very far out)
There’s a lot to say about why I think SC will probably happen in the next five years or so. I’ll go leave line-by-line comments in the relevant section of your post!
Yeah, someone pointed out that footnote to me, and I laughed a bit. It’s very small and easy to miss. I don’t think you guys actually misrepresented anything. It’s clear from reading the research section what your actual timelines are and so on. I’m just pointing to communication issues.
Talking about 2027, the authors did inform the readers in a footnote, but revisions of the timelines forecast turned out to be hard to deliver to the general public. Let’s wait for @Daniel Kokotajlo to state his opinion on the doubts related to SOTA architecture. In my opinion these problems would be resolved by a neuralese architecture or an architecture which could be an even bigger breakthrough (neuralese with big internal memory?)
Hey Daniel, I loved the podcast with Dwarkesh and Scott Alexander. I am glad you have gotten people talking about this, though I’m of two minds about it, because as I say in my post, I believe your estimates in the AI 2027 document are very aggressive (and there were some communication issues there I discussed with Eli in another comment). I worry what might happen in 2028 if basically the entire scenario described on the main page turns out to not happen, which is what I believe.
My blog post is a reaction to the AI 2027 document as it stands, which doesn’t seem to have any banner at the top saying the authors no longer endorse the findings or anything. The domain name and favicon haven’t changed. I am now aware that you have adjusted your median timeline from 2027 to 2028, and that Eli has adjusted his estimate as well. The scenario itself describes some pretty crazy things happening in 2027, and the median estimate in the research (before adjustments) seems to be 2027 for SC, SAR, SAIR, and ASI all in 2027. I definitely don’t agree that any of those milestones will be reached in 2027, nor in 2028.
Of course, predicting the future is hard the further out in time we go. I have a strongish sense for 2027, but I would put SC, SAR, SAIR, and ASI all at least ten years further than that, and it’s really hard to know what the heck will be going on in the world by then, because unlike you I don’t believe this is a matter of continuous progress (for reasons mentioned in my post), and discontinuous progress is hard to predict. In 1902 Simon Newcomb said, “Flight by machines heavier than air is unpractical and insignificant, if not utterly impossible.” The next year the Wright brothers took off. I’m not sure trying to put a number on it is better than simply saying, “I don’t know.”
In the spirit of answering the question you asked, I’d predict SC in the year 2050. But this is such a low-confidence prediction as to be essentially worthless, like saying I think there’s a 50% God exists.
I’d like to know what you thought of my justification for my doubts about LLMs, if you had time to get that far.
Thanks for the critique & the reply btw! Very much appreciate you giving quantitative alternative credences to mine, it’s a productive way to focus the conversation I think.
Footnote #1 used to be attached to “We wrote a scenario...” but we wanted it to be more prominent in response to exactly the sort of criticism you are making, so we moved it up to be literally a footnote on the title. I suppose we could have put it in the main text itself.
My median is currently 2029 actually; at the time AI 2027 was published it was 2028.
OK, thanks, that’s helpful. So yeah, while we agree that SC probably won’t happen by 2027 EOY, we do still have a disagreement—I think it probably WILL happen in the next five years or so (and the rest of the team thinks it’ll probably happen in the next ten years or so) whereas you seem confident it WON’T happen before 2037. I hope you are right! I agree also that the future is very hard to predict, especially the farther out it is (and 2037+ is very far out)
There’s a lot to say about why I think SC will probably happen in the next five years or so. I’ll go leave line-by-line comments in the relevant section of your post!
Yeah, someone pointed out that footnote to me, and I laughed a bit. It’s very small and easy to miss. I don’t think you guys actually misrepresented anything. It’s clear from reading the research section what your actual timelines are and so on. I’m just pointing to communication issues.
Thanks for your responses! I’ll check them out.
Talking about 2027, the authors did inform the readers in a footnote, but revisions of the timelines forecast turned out to be hard to deliver to the general public. Let’s wait for @Daniel Kokotajlo to state his opinion on the doubts related to SOTA architecture. In my opinion these problems would be resolved by a neuralese architecture or an architecture which could be an even bigger breakthrough (neuralese with big internal memory?)