I also think that there is a <5% chance that a large-scale AI catastrophe occurs in the next 6 months, but I don’t think the time horizon framing here is a compelling argument for that.
Let’s use one year of human labor as a lower bound on how difficult it is. This means that AI systems will need to at least have a time horizon of one work-year (2000 hours) in order to cause a catastrophe.
I’m not very sold by this argument. AIs can do many tasks that take over a year of human labor, like “write 1 billion words of fiction”; when thinking about time horizons we care about something like “irreducible time horizon”, or some appeal to the most difficult or long-horizon-y bottleneck within a task.
While I agree that causing a catastrophe would require more than a year of human labor, it is not obvious to me that causing a large-scale catastrophe is bottlenecked on any task with an irreducible time horizon of one year; indeed, it’s not obvious to me that any such tasks exist! It seems plausible to me that the first AIs with reliable 1-month time horizons will basically not be time-horizon-limited in any way that humans aren’t, and unlikely-but-possible that this will be true even at the 1 week level.
Concretely, massively scaled coordinated cyberattacks on critical infrastructure worldwide is a threat model that could plausibly cause 1e8 deaths and does not obviously rely on any subtask with an especially large time horizon; I think the primary blockers to autonomous LLM “success” here in 2025 are (1) programming skill (2) ability to make and execute on an okay plan coherently enough not to blunder into failure (3) ability to acquire enough unmonitored inference resources (4) alignment. Of these I expect (2) to be the most time-horizon bottlenecked, but I wouldn’t feel that surprised if models that could pull it off still had low reliability at 1-day AI R&D tasks. (This particular scenario is contingent enough that I think it’s still very unlikely in the near term, TBC.)
AIs with reliable 1-month time horizons will basically not be time-horizon-limited in any way that humans aren’t
In this statement, are you thinking about time horizons as operationalized / investigated in the METR paper, or are you thinking about the True Time Horizon?
More like True Time Horizon, though I think the claim is pretty plausible within the domain of well-scoped end-to-end AI R&D remote work tasks as well.
I also think that there is a <5% chance that a large-scale AI catastrophe occurs in the next 6 months, but I don’t think the time horizon framing here is a compelling argument for that.
I’m not very sold by this argument. AIs can do many tasks that take over a year of human labor, like “write 1 billion words of fiction”; when thinking about time horizons we care about something like “irreducible time horizon”, or some appeal to the most difficult or long-horizon-y bottleneck within a task.
While I agree that causing a catastrophe would require more than a year of human labor, it is not obvious to me that causing a large-scale catastrophe is bottlenecked on any task with an irreducible time horizon of one year; indeed, it’s not obvious to me that any such tasks exist! It seems plausible to me that the first AIs with reliable 1-month time horizons will basically not be time-horizon-limited in any way that humans aren’t, and unlikely-but-possible that this will be true even at the 1 week level.
Concretely, massively scaled coordinated cyberattacks on critical infrastructure worldwide is a threat model that could plausibly cause 1e8 deaths and does not obviously rely on any subtask with an especially large time horizon; I think the primary blockers to autonomous LLM “success” here in 2025 are (1) programming skill (2) ability to make and execute on an okay plan coherently enough not to blunder into failure (3) ability to acquire enough unmonitored inference resources (4) alignment. Of these I expect (2) to be the most time-horizon bottlenecked, but I wouldn’t feel that surprised if models that could pull it off still had low reliability at 1-day AI R&D tasks. (This particular scenario is contingent enough that I think it’s still very unlikely in the near term, TBC.)
In this statement, are you thinking about time horizons as operationalized / investigated in the METR paper, or are you thinking about the True Time Horizon?
More like True Time Horizon, though I think the claim is pretty plausible within the domain of well-scoped end-to-end AI R&D remote work tasks as well.