I notice that I am confused. You imply that the human-equivalent horizon of a model is Hh=2αh−αβ−βh Then the LOGARITHM of Hh is αh−αβ−βhand it is the LOGARITHM which likely behaves linearly if β is constant and α changes linearly or hyperbolically if β changes linearly. Alas, β doesn’t change linearly across models. Instead, as far as I understand, β is calculated as (logTH50TH80)−1 Were β monotoneous, we would also expect monotoneous changes in the ratio of time horizons TH50TH80 Instead, the ratios are this. Setting aside Claude Opus 4.5 with the ratio equal to 10.64, the next two biggest ratios are displayed by DeepSeek R1-0528 (8.53) and Grok 4 (7.14). Therefore, the ratio of the time horizons did NOT display a consistent trend at least before Claude Opus 4.5.
If you run a linear regression on β versus time, then the regression line does have a positive slope estimate (even pre-claude-4.5), and I used that regression line in section 2.2.1.1 to provide an alternative estimate for when LLMs will catch up with human β (at which point the denominator goes to zero, and the LLM’s horizon blows up). That said, the β trend is quite noisy; and while the OpenAI trend from that estimate was already stat sig, the overall β trend across companies did not hit stat sig until the Claude Opus 4.5 datapoint. Also, as I argued in the post (partly due to the noisy β trend): “I suspect it makes more sense to directly extrapolate the overall time-horizon estimate rather than linearly extrapolating the noisy logistic coefficient in isolation, even if the slope trend is a useful intuition-pump for seeing why a finite-time blowup is plausible” (btw, the specific projection in that section was based on just OpenAI instruct-tuned models, i.e. post gpt-3 models)
Also, the underlying question in this case was whether the LLM slope would catch up to human baseline slope, but it’s somewhat moot since that has basically happened with Claude 4.5; and if METR were to collect better incentivized human baselines (with better slope β), it seems quite likely that there would be a similar catch up to match this improved β, leading to another blowup in the updated (human-relative) LLM time horizons.
Neither I nor GPT-5.2 believe that THIS trend is consistent enough. Additionally, Claude Opus 4.5 had itsshare of doubts cast upon the abnormally high 50% time horizon. Finally, what would it mean for a hired human to have a 50% or 80% chance of succeeding at year-long tasks? That the human cannot do the task ~at all, even given 10 years? But even this example is not that an example...
Your link to chatgpt’s analysis is not loading for me, but I don’t really trust chatgpt to do this without mistakes anyway; when I run this regression in python statsmodels with METR’s official β values and release dates from their github (plus the recent gpt-5 models), it is stat sig for this OpenAI instruct-model trend (p ~ .04) and it gets more stat sig once we include Claude 4.5. I don’t see what dates you are using, but your beta values have a fair amount of (rounding?) error compared to METR’s data, and the model list isn’t the same, e.g. they didn’t include non-frontier mini models, etc. (btw, note that one-sided p-values are appropriate in this case since only positive slope would be treated as evidence of catching up).
But again, the main question from that particular trend/argument was whether LLMs were catching up with the human baseline slope, which Claude now has basically done for the existing baselines, so that seems pretty strongly confirmatory. It’s true that Claude has a somewhat worse logistic intercept than gpt-5 models (associated w/ a bit worse short-task reliability), but it is still better than the human baseline intercept, and the net effect is that Claude beats the human baselines over a much longer range of horizons than other models (per METR’s logistic fits).
As far as what it would mean for humans to not be able to complete really long/difficult tasks at high reliability, I wonder if that comes back to this argument from the post:
“Many people have the intuition that humans can handle tasks of arbitrary length at high reliability, but of course that depends on task difficulty, and while we can extrapolate the METR curve to weeks or months, the existing/actual tasks are short, so it’s not clear how to estimate the difficulty of hypothetical long METR tasks. There is a tendency to assume these would just be typical long software engineering tasks (e.g. merely time-consuming due to many fairly straightforward subtasks), but there is not much basis for that assumption, as opposed to longer tasks on this length/difficulty trend being more like ‘prove this tricky mathematical theorem’, etc”
I notice that I am confused. You imply that the human-equivalent horizon of a model is Hh=2αh−αβ−βh Then the LOGARITHM of Hh is αh−αβ−βhand it is the LOGARITHM which likely behaves linearly if β is constant and α changes linearly or hyperbolically if β changes linearly. Alas, β doesn’t change linearly across models. Instead, as far as I understand, β is calculated as (logTH50TH80)−1 Were β monotoneous, we would also expect monotoneous changes in the ratio of time horizons TH50TH80 Instead, the ratios are this. Setting aside Claude Opus 4.5 with the ratio equal to 10.64, the next two biggest ratios are displayed by DeepSeek R1-0528 (8.53) and Grok 4 (7.14). Therefore, the ratio of the time horizons did NOT display a consistent trend at least before Claude Opus 4.5.
If you run a linear regression on β versus time, then the regression line does have a positive slope estimate (even pre-claude-4.5), and I used that regression line in section 2.2.1.1 to provide an alternative estimate for when LLMs will catch up with human β (at which point the denominator goes to zero, and the LLM’s horizon blows up). That said, the β trend is quite noisy; and while the OpenAI trend from that estimate was already stat sig, the overall β trend across companies did not hit stat sig until the Claude Opus 4.5 datapoint. Also, as I argued in the post (partly due to the noisy β trend): “I suspect it makes more sense to directly extrapolate the overall time-horizon estimate rather than linearly extrapolating the noisy logistic coefficient in isolation, even if the slope trend is a useful intuition-pump for seeing why a finite-time blowup is plausible” (btw, the specific projection in that section was based on just OpenAI instruct-tuned models, i.e. post gpt-3 models)
Also, the underlying question in this case was whether the LLM slope would catch up to human baseline slope, but it’s somewhat moot since that has basically happened with Claude 4.5; and if METR were to collect better incentivized human baselines (with better slope β), it seems quite likely that there would be a similar catch up to match this improved β, leading to another blowup in the updated (human-relative) LLM time horizons.
(edited)
The trend itself was this:
gpt_3_5_turbo_instruct 3.49
gpt_4 5.54
gpt_4_0125 4.47
gpt_4_1106 5.87
gpt_4_turbo 4.30
gpt_4o 5.48
o1_preview 4.76
o1_elicited 6.76
o3 4.39
o4-mini 4.93
gpt_5 5.18
gpt_5_1_codex_max 5.36
Neither I nor GPT-5.2 believe that THIS trend is consistent enough. Additionally, Claude Opus 4.5 had its share of doubts cast upon the abnormally high 50% time horizon. Finally, what would it mean for a hired human to have a 50% or 80% chance of succeeding at year-long tasks? That the human cannot do the task ~at all, even given 10 years? But even this example is not that an example...
Your link to chatgpt’s analysis is not loading for me, but I don’t really trust chatgpt to do this without mistakes anyway; when I run this regression in python statsmodels with METR’s official β values and release dates from their github (plus the recent gpt-5 models), it is stat sig for this OpenAI instruct-model trend (p ~ .04) and it gets more stat sig once we include Claude 4.5. I don’t see what dates you are using, but your beta values have a fair amount of (rounding?) error compared to METR’s data, and the model list isn’t the same, e.g. they didn’t include non-frontier mini models, etc. (btw, note that one-sided p-values are appropriate in this case since only positive slope would be treated as evidence of catching up).
But again, the main question from that particular trend/argument was whether LLMs were catching up with the human baseline slope, which Claude now has basically done for the existing baselines, so that seems pretty strongly confirmatory. It’s true that Claude has a somewhat worse logistic intercept than gpt-5 models (associated w/ a bit worse short-task reliability), but it is still better than the human baseline intercept, and the net effect is that Claude beats the human baselines over a much longer range of horizons than other models (per METR’s logistic fits).
As far as what it would mean for humans to not be able to complete really long/difficult tasks at high reliability, I wonder if that comes back to this argument from the post:
“Many people have the intuition that humans can handle tasks of arbitrary length at high reliability, but of course that depends on task difficulty, and while we can extrapolate the METR curve to weeks or months, the existing/actual tasks are short, so it’s not clear how to estimate the difficulty of hypothetical long METR tasks. There is a tendency to assume these would just be typical long software engineering tasks (e.g. merely time-consuming due to many fairly straightforward subtasks), but there is not much basis for that assumption, as opposed to longer tasks on this length/difficulty trend being more like ‘prove this tricky mathematical theorem’, etc”
(edited)