As a minor point of feedback, I’d suggest adding a bit of material near the top of the timelines and/or takeoff forecasts, clarifying the range of activities meant to be included in “superhuman coder” and “superhuman AI researcher”, e.g. listing some activities that are and are not in scope. I was startled to see Ryan say “my sense is that an SAR has to be better than humans at basically everything except vision”; I would never have guessed that was the intended interpretation.)
This is fair. To the extent we have chosen what activities to include, it’s supposed to encompass everything that any researcher/engineer currently does to improve AIs’ AI R&D capabilities within AGI companies, see the AI R&D progress multiplier definition: “How much faster would AI R&D capabilities...”. As to whether we should include activities that researchers or engineers don’t do, my instinct is mostly no because the main thing I can think of there is data collection, and that feels like it should be treated separately (in the AI R&D progress multiplier appendix, we clarify that using new models for synthetic data generation isn’t included in the AI R&D progress multiplier as we want to focus on improved research skills, though I’m unsure if that the right choice and am open to changing).
But I did not put a lot of effort into thinking about how exactly to define the range of applicable activities and what domains should be included; My intuition is that it matters less than you think because I expect automation to be less jagged than you (I might write more about that in a separate comment) and because of intuitions that research taste is the key skill and is relatively domain-general, though I agree expertise helps. I agree that there will be varying multipliers depending on the domain, but given that the takeoff forecast is focused mostly on a set of AI R&D-specific milestones, I think it makes sense to focus on that.
This is fair. To the extent we have chosen what activities to include, it’s supposed to encompass everything that any researcher/engineer currently does to improve AIs’ AI R&D capabilities within AGI companies, see the AI R&D progress multiplier definition: “How much faster would AI R&D capabilities...”. As to whether we should include activities that researchers or engineers don’t do, my instinct is mostly no because the main thing I can think of there is data collection, and that feels like it should be treated separately (in the AI R&D progress multiplier appendix, we clarify that using new models for synthetic data generation isn’t included in the AI R&D progress multiplier as we want to focus on improved research skills, though I’m unsure if that the right choice and am open to changing).
But I did not put a lot of effort into thinking about how exactly to define the range of applicable activities and what domains should be included; My intuition is that it matters less than you think because I expect automation to be less jagged than you (I might write more about that in a separate comment) and because of intuitions that research taste is the key skill and is relatively domain-general, though I agree expertise helps. I agree that there will be varying multipliers depending on the domain, but given that the takeoff forecast is focused mostly on a set of AI R&D-specific milestones, I think it makes sense to focus on that.