OpenAI released a detailed write-up on research acceleration. It includes both research acceleration estimates (experiments per engineer) and model time horizon estimates (METR style).
With rough Fable analysis, the “Agents are increasingly solving more complex tasks for researchers” chart is showing METR-logistic curve time horizon doubling rates of 3.5-4 months at 50% and 80% accuracy (when throwing out outliers of January and June), with no visible shortening of doubling time.
(As usual this is very noisy and the 80⁄50 horizon ratio being 12x+ compared to METR’s ~6x suggests this isn’t necessarily comparable)
OpenAI released a detailed write-up on research acceleration. It includes both research acceleration estimates (experiments per engineer) and model time horizon estimates (METR style).
With rough Fable analysis, the “Agents are increasingly solving more complex tasks for researchers” chart is showing METR-logistic curve time horizon doubling rates of 3.5-4 months at 50% and 80% accuracy (when throwing out outliers of January and June), with no visible shortening of doubling time.
(As usual this is very noisy and the 80⁄50 horizon ratio being 12x+ compared to METR’s ~6x suggests this isn’t necessarily comparable)
Linkpost here.