I’ve seen demonstrations of AIs accomplishing [...] tasks that would take humans months to years. Due to this, I tentatively believe that (as of March 1st) the well-elicited 50% reliability time-horizon on ESNI tasks (using only publicly available models) is somewhere between a month and several years
Once you are talking about scaling to arbitrary inference costs, I think the relevant notion of time horizon is closer to: “For what T could you solve this task using an arbitrary supply of humans each of whom only works on the project for T hours?”
On this framing I think that lots of easily-checkable tasks have much shorter horizons than “amount of time they would take a human to do.” That is, I think that you could successfully solve them by delegating to a giant team of low-context humans each of whom makes marginal improvements, and the work from those humans would look quite similar to the AI work.
This is particularly true for reimplementation tasks with predefined tests + a reference implementation. In that setting a human can pick a specific failing test, use the reference implementation to understand the desired behavior, and then implement it while using tests to make sure they don’t break anything else. For the projects you are describing it seems plausible that a human with a few days could use that workflow to reliably make forward progress and get the project done in a moderate number of steps.
(I would expect a giant human team using that methodology to make a huge number of mistakes for anything not covered by the tests, and to write code that is low quality and hard to maintain. But I think we’re currently seeing roughly that pattern of failures from AI reimplementation.)
It’s also very true of vulnerability discovery, since different humans can look in different places and then iteratively zoom in on whatever leads are most promising.
So I’m not sure how much of the phenomenon you’re observe is “there is a way longer horizon for these tasks” rather than “a more careful definition of horizon is more important for these tasks.” Probably some of both, but It seems quite possible that for the tasks you are describing the horizon length is really more like a few days or a week than months or a year.
I think it’s helpful to try to pull these effects apart, particularly when reasoning about the likely effects of continuing RL schlep: I think that you are seeing good performance on these tasks partly because they are easy to decompose and delegate, and not entirely because AI developers are able to do RL on them. For the very “long horizon” tasks you cite, I think that’s the dominant effect. So I wouldn’t expect performance to be as impressive on tasks where it would be hard to delegate them to a giant team of low-context humans.
I still agree that recent public development seem like evidence for shorter timelines, and I’m not saying my bottom line numbers differ much from yours. Even a more conservative extrapolation of time horizons would suggest more like 2-4 years until you can strictly outperform humans for virtual projects (with human parity somewhat before that).
This is a good point and I broadly agree with what you’re saying. Some possible disagreements:
Once you are talking about scaling to arbitrary inference costs, I think the relevant notion of time horizon is closer to: “For what T could you solve this task using an arbitrary supply of humans each of whom only works on the project for T hours?”
Maybe I should have made this clearer, but I’m not talking about scaling to arbitrary inference costs. Instead, I’m talking about scaling to inference costs that are a moderate fraction of the human task completion cost. (E.g., 1%-100% depending on the task.) I think you’d want to compare the AI performance at some inference budget to human labor with some limit in supply.
So I’m not sure how much of the phenomenon you’re observe is “there is a way longer horizon for these tasks” rather than “a more careful definition of horizon is more important for these tasks.” Probably some of both, but It seems quite possible that for the tasks you are describing the horizon length is really more like a few days or a week than months or a year.
Yep, it seems reasonably likely that on this alternative notion of time horizon, the horizon length is more like a few days. However, under this alternative definition, relatively small time-horizon values may correspond to much larger (real-world) impact. For example, an AI with a decomposed-time-horizon of ~1 week could potentially speed up AI R&D a lot, while under the original time-horizon notion, a 1-week 50%-reliability time horizon is much less of a big deal. (To be more precise about the decomposed-time-horizon metric: this would correspond to AI having a 50% chance of matching a team of humans where each human gets 1 week before being replaced, with total labor-hours capped at ~100x what the task would normally require to avoid degeneracies from truly arbitrary inference spend.) So the key question becomes: how much can you accomplish if the task must be heavily decomposed but you can apply much more labor? I’m pretty uncertain about this for the tasks we care about.
Further, AIs may soon end up being especially good (superhuman?) at working in heavily decomposed contexts—e.g., good at leaving notes to other instances, good at quickly picking up project state from limited context. This makes the correspondence to doing task decomposition with humans more fraught, even though AIs will still do relatively better at tasks that are easier to do incrementally or otherwise decompose. I think it’s already the case that AIs are extremely good relative to humans at loading up complex state so long as that state is written down (reasonably clearly).
I was already pretty uncertain about how to translate time horizon into downstream impacts, and while this alternative metric might be more consistent across different groups of tasks, it makes translating to downstream impact harder. An alternative is just explicitly thinking about the time horizon for different buckets of tasks using the original notion.
Regardless, I agree that AI capabilities may be better understood using this alternative time-horizon notion, and this does closely correspond to how AIs are actually being used. (My scaffold is mostly just doing task decomposition, and something like this is effectively required for good performance given current AI properties.)
Maybe I should have made this clearer, but I’m not talking about scaling to arbitrary inference costs. Instead, I’m talking about scaling to inference costs that are a moderate fraction of the human task completion cost. (E.g., 1%-100% depending on the task.) I think you’d want to compare the AI performance at some inference budget to human labor with some limit in supply.
I agree. In the more realistic regime you are talking about you have some more complicated quantitative question around how large are the slowdowns from task decomposition into what scale.
My main point was that for the tasks we are talking about here, the slowdowns seem like they might not be that large even for modest human time horizons. (In contrast with some of the crazy factored cognition stuff we have sometimes talked about, which involves much shorter horizons, much harder-to-decompose tasks, and much larger slowdowns.)
However, under this alternative definition, relatively small time-horizon values may correspond to much larger (real-world) impact.
I agree that this could lead to large impacts with relatively short horizons (perhaps even today’s horizons, with an appropriately broadened training distribution and a bunch of schlep). That does imply a different picture of AI strengths and weaknesses (e.g. weaker on-the-job learning with performance mostly limited to domains near the training distribution; differential speedup for tasks that are easily decomposed), with a more schlep-heavy singularity, a greater role for tight human involvement later in the process, and probably less alignment concern earlier in the trajectory.
Once you are talking about scaling to arbitrary inference costs, I think the relevant notion of time horizon is closer to: “For what T could you solve this task using an arbitrary supply of humans each of whom only works on the project for T hours?”
On this framing I think that lots of easily-checkable tasks have much shorter horizons than “amount of time they would take a human to do.” That is, I think that you could successfully solve them by delegating to a giant team of low-context humans each of whom makes marginal improvements, and the work from those humans would look quite similar to the AI work.
This is particularly true for reimplementation tasks with predefined tests + a reference implementation. In that setting a human can pick a specific failing test, use the reference implementation to understand the desired behavior, and then implement it while using tests to make sure they don’t break anything else. For the projects you are describing it seems plausible that a human with a few days could use that workflow to reliably make forward progress and get the project done in a moderate number of steps.
(I would expect a giant human team using that methodology to make a huge number of mistakes for anything not covered by the tests, and to write code that is low quality and hard to maintain. But I think we’re currently seeing roughly that pattern of failures from AI reimplementation.)
It’s also very true of vulnerability discovery, since different humans can look in different places and then iteratively zoom in on whatever leads are most promising.
So I’m not sure how much of the phenomenon you’re observe is “there is a way longer horizon for these tasks” rather than “a more careful definition of horizon is more important for these tasks.” Probably some of both, but It seems quite possible that for the tasks you are describing the horizon length is really more like a few days or a week than months or a year.
I think it’s helpful to try to pull these effects apart, particularly when reasoning about the likely effects of continuing RL schlep: I think that you are seeing good performance on these tasks partly because they are easy to decompose and delegate, and not entirely because AI developers are able to do RL on them. For the very “long horizon” tasks you cite, I think that’s the dominant effect. So I wouldn’t expect performance to be as impressive on tasks where it would be hard to delegate them to a giant team of low-context humans.
I still agree that recent public development seem like evidence for shorter timelines, and I’m not saying my bottom line numbers differ much from yours. Even a more conservative extrapolation of time horizons would suggest more like 2-4 years until you can strictly outperform humans for virtual projects (with human parity somewhat before that).
This is a good point and I broadly agree with what you’re saying. Some possible disagreements:
Maybe I should have made this clearer, but I’m not talking about scaling to arbitrary inference costs. Instead, I’m talking about scaling to inference costs that are a moderate fraction of the human task completion cost. (E.g., 1%-100% depending on the task.) I think you’d want to compare the AI performance at some inference budget to human labor with some limit in supply.
Yep, it seems reasonably likely that on this alternative notion of time horizon, the horizon length is more like a few days. However, under this alternative definition, relatively small time-horizon values may correspond to much larger (real-world) impact. For example, an AI with a decomposed-time-horizon of ~1 week could potentially speed up AI R&D a lot, while under the original time-horizon notion, a 1-week 50%-reliability time horizon is much less of a big deal. (To be more precise about the decomposed-time-horizon metric: this would correspond to AI having a 50% chance of matching a team of humans where each human gets 1 week before being replaced, with total labor-hours capped at ~100x what the task would normally require to avoid degeneracies from truly arbitrary inference spend.) So the key question becomes: how much can you accomplish if the task must be heavily decomposed but you can apply much more labor? I’m pretty uncertain about this for the tasks we care about.
Further, AIs may soon end up being especially good (superhuman?) at working in heavily decomposed contexts—e.g., good at leaving notes to other instances, good at quickly picking up project state from limited context. This makes the correspondence to doing task decomposition with humans more fraught, even though AIs will still do relatively better at tasks that are easier to do incrementally or otherwise decompose. I think it’s already the case that AIs are extremely good relative to humans at loading up complex state so long as that state is written down (reasonably clearly).
I was already pretty uncertain about how to translate time horizon into downstream impacts, and while this alternative metric might be more consistent across different groups of tasks, it makes translating to downstream impact harder. An alternative is just explicitly thinking about the time horizon for different buckets of tasks using the original notion.
Regardless, I agree that AI capabilities may be better understood using this alternative time-horizon notion, and this does closely correspond to how AIs are actually being used. (My scaffold is mostly just doing task decomposition, and something like this is effectively required for good performance given current AI properties.)
I agree. In the more realistic regime you are talking about you have some more complicated quantitative question around how large are the slowdowns from task decomposition into what scale.
My main point was that for the tasks we are talking about here, the slowdowns seem like they might not be that large even for modest human time horizons. (In contrast with some of the crazy factored cognition stuff we have sometimes talked about, which involves much shorter horizons, much harder-to-decompose tasks, and much larger slowdowns.)
I agree that this could lead to large impacts with relatively short horizons (perhaps even today’s horizons, with an appropriately broadened training distribution and a bunch of schlep). That does imply a different picture of AI strengths and weaknesses (e.g. weaker on-the-job learning with performance mostly limited to domains near the training distribution; differential speedup for tasks that are easily decomposed), with a more schlep-heavy singularity, a greater role for tight human involvement later in the process, and probably less alignment concern earlier in the trajectory.