Thanks for the correction! I’m guessing you don’t want to, but I would appreciate an elaboration on your part; is @habryka’s description below inaccurate, or did I misinterpret it?
It’s just like, a semi-random methodology for forecasting a transformative AI timeline that vaguely informed the main scenario. Conceptually, it feels similar to just doing a random fermi estimate in the middle of a blog post to sanity-check that the thing I am thinking about isn’t completely crazy.
OK I just had a chat with Eli to try to trace the causal history as best we can remember. At a high level, we were working on the scenario and the supplementary research in parallel, and went back and forth making edits to both for months, and our views evolved somewhat over the course of that time.
Timelines: We initially set AGI in 2027 based on my AGI median, which was informed based on a combination of arguments regarding gains from scaling up agency training, as well as a very crude, handwavy version of what later became the benchmarks and gaps model. Later timelines modeling (the stuff that actually went on the website) along with some additional evidence that came out, pushed my median back to 2028. We denoted this in a footnote on the site (footnote #1 in fact) and I posted a shortform about it (plus a tweet or two). tl;dr is that 2027 was my mode, not my median, after the update. We considered rewriting the scenario to happen about one year later, due to this, but decided against since that would have taken a lot of extra time and didn’t really change any of the implications. If the timelines model had given very different results which changed our views against 2027 being plausible, then we would have re-written the scenario. I also mentioned this to Kevin Roose in my interview with him (my somewhat later timelines, the difference between median and mode). I didn’t expect people to make such a big deal of this.
Takeoff: The takeoff model for our first scenario, the “practice scenario” which we basically scrapped, was basically a simplified version of Davidson’s takeoff speeds model. (takeoffspeeds.com) Later takeoff modeling informed which milestones to focus on the scenario (superhuman coder, superhuman AI researcher, etc.) and what AI R&D progress multiplier they should have. Our memory isn’t clear on to what extent they also resulted in changes to the speed of the milestone progression. We think an early crude version of our takeoff model might have resulted in significant changes, but we aren’t sure. We were also working on our takeoff model up until the last minute, and similar to the timelines model mostly used it as a sanity check.
Compute: The first version of this was done in early 2024, and the result of it and future versions were directly imported into the scenario.
AI Goals: Early versions of this supplement were basically responsible for our decision to go with instrumentally convergent goals as the AIs’ ultimate goals in the scenario.
Security: This one was in between a sanity check and directly feeding into the scenario. It didn’t result in large changes but confirmed the likelihood of the weight theft and informed various decisions about e.g. cyberattacks.
So.… Habryka’s description is somewhat accurate, certainly more accurate than your description (“no meaningful sense”). But I think it still undersells it. That said, it’s definitely not the case that we wrote all the supplements first and then wrote the scenario based on the outputs of those calculations; instead, we wrote them in parallel, had various shitty early versions, etc.
If you want to know more about the evidence & modelling that shaped our views in early 2024 when we were starting the project, I could try to compile a list. I’ve already mentioned takeoffspeeds.com for example. There’s lots of other writing I’ve put on LessWrong on the subject as well.
My guess is there is no confusion about this, but to be clear, I didn’t intend to speak on behalf of the AI 2027 team. Indeed, it’s plausible to me they disagree with it, though my honest belief in that case is that they are confused about the sources of their own beliefs, not that my statement is wrong. I.e. I said:
Ideally the research supplements would say something like that at the top, though it’s plausible that some of the AI Futures Project team relate to their epistemic process differently (though if they do, I think they are just kind of confused).
Thanks for the correction! I’m guessing you don’t want to, but I would appreciate an elaboration on your part; is @habryka’s description below inaccurate, or did I misinterpret it?
OK I just had a chat with Eli to try to trace the causal history as best we can remember. At a high level, we were working on the scenario and the supplementary research in parallel, and went back and forth making edits to both for months, and our views evolved somewhat over the course of that time.
Timelines: We initially set AGI in 2027 based on my AGI median, which was informed based on a combination of arguments regarding gains from scaling up agency training, as well as a very crude, handwavy version of what later became the benchmarks and gaps model. Later timelines modeling (the stuff that actually went on the website) along with some additional evidence that came out, pushed my median back to 2028. We denoted this in a footnote on the site (footnote #1 in fact) and I posted a shortform about it (plus a tweet or two). tl;dr is that 2027 was my mode, not my median, after the update. We considered rewriting the scenario to happen about one year later, due to this, but decided against since that would have taken a lot of extra time and didn’t really change any of the implications. If the timelines model had given very different results which changed our views against 2027 being plausible, then we would have re-written the scenario. I also mentioned this to Kevin Roose in my interview with him (my somewhat later timelines, the difference between median and mode). I didn’t expect people to make such a big deal of this.
Takeoff: The takeoff model for our first scenario, the “practice scenario” which we basically scrapped, was basically a simplified version of Davidson’s takeoff speeds model. (takeoffspeeds.com) Later takeoff modeling informed which milestones to focus on the scenario (superhuman coder, superhuman AI researcher, etc.) and what AI R&D progress multiplier they should have. Our memory isn’t clear on to what extent they also resulted in changes to the speed of the milestone progression. We think an early crude version of our takeoff model might have resulted in significant changes, but we aren’t sure. We were also working on our takeoff model up until the last minute, and similar to the timelines model mostly used it as a sanity check.
Compute: The first version of this was done in early 2024, and the result of it and future versions were directly imported into the scenario.
AI Goals: Early versions of this supplement were basically responsible for our decision to go with instrumentally convergent goals as the AIs’ ultimate goals in the scenario.
Security: This one was in between a sanity check and directly feeding into the scenario. It didn’t result in large changes but confirmed the likelihood of the weight theft and informed various decisions about e.g. cyberattacks.
So.… Habryka’s description is somewhat accurate, certainly more accurate than your description (“no meaningful sense”). But I think it still undersells it. That said, it’s definitely not the case that we wrote all the supplements first and then wrote the scenario based on the outputs of those calculations; instead, we wrote them in parallel, had various shitty early versions, etc.
If you want to know more about the evidence & modelling that shaped our views in early 2024 when we were starting the project, I could try to compile a list. I’ve already mentioned takeoffspeeds.com for example. There’s lots of other writing I’ve put on LessWrong on the subject as well.
Does this help?
My guess is there is no confusion about this, but to be clear, I didn’t intend to speak on behalf of the AI 2027 team. Indeed, it’s plausible to me they disagree with it, though my honest belief in that case is that they are confused about the sources of their own beliefs, not that my statement is wrong. I.e. I said: