But it isn’t August or September yet. Maybe someone will end up actually creating capable agents. In addition, the amount of operations used for creating Grok 4 was estimated as 4e27--6e27, which seems to align with the forecast. The research boost rate by Grok 4 or a potentially tweaked model wasn’t estimated. Maybe Grok 4 or an AI released in August will boost research speed?
It was indicated in the opening slide of Grok 4 release livestream that Grok 4 was pretrained with the same amount of compute as Grok 3, which in turn was pretrained on 100K H100s, so probably 3e26 FLOPs (40% utilization for 3 months with 1e15 FLOP/s per chip). RLVR has a 3x-4x lower compute utilization than pretraining, so if we are insisting on counting RLVR in FLOPs, then 3 months of RLVR might be 9e25 FLOPs, for the total of 4e26 FLOPs.
Stargate Abilene will be 400K chips in GB200 NVL72 racks in 2026, which is 10x more FLOP/s than 100K H100s. So it’ll be able to train 4e27-8e27 FLOPs models (pretraining and RLVR, in 3+3 months), and it might be early 2027 when they are fully trained. (Google is likely to remain inscrutable in their training compute usage, though Meta might also catch up by then.)
the amount of operations used for creating Grok 4 was estimated as 4e27--6e27
(I do realize it’s probably some sort of typo, either yours or in your unnamed source. But 10x is almost 2 years of even the current fast funding-fueled scaling, that’s not a small difference.)
We’ve been going on back and forth on this a bit—it seems like your model suggests AGI in 2027 is pretty unlikely?
That is, we see the first generation of massively scaled RLVR around 2026/2027. So it kind of has to work out of the box for AGI to arrive that quickly?
I suppose this is just speculation though. Maybe it’s useful enough that the next generation is somehow much, much faster to arrive?
That is, we see the first generation of massively scaled RLVR around 2026/2027. So it kind of has to work out of the box for AGI to arrive that quickly?
By 2027, we’ll also have 10x scaled-up pretraining compared to current models (trained on 2024 compute). And correspondingly scaled RLVR, with many diverse tool-using environments that are not just about math and coding contest style problems. If we go 10x lower than current pretraining, we get original GPT-4 from Mar 2023, which is significantly worse than the current models. So with 10x higher pretraining than current models, the models of 2027 might make significantly better use of RLVR training than the current models can.
Also, 2 years might be enough time to get some sort of test-time training capability started, either with novel or currently-secret methods, or by RLVRing models to autonomously do post-training on variants of themselves to make them better at particular sources of tasks during narrow deployment. Apparently Sutskever’s SSI is rumored to be working on the problem (at 39:25 in the podcast), and overall this seems like the most glaring currently-absent faculty. (Once it’s implemented, something else might end up a similarly obvious missing piece.)
it seems like your model suggests AGI in 2027 is pretty unlikely?
I’d give it 10% (for 2025-2027). From my impression of the current capabilities and the effect of scaling so far, the remaining 2 OOMs of compute seem like a 30% probability of getting there (by about 2030), with a third of it in the first 10x of the remaining scaling, that is 10% with 2026 compute (for 2027 models). After 2029, scaling slows down to a crawl (relatively speaking), so maybe another 50% for the 1000x of scaling in 2030-2045 when there’ll also be time for any useful schlep, with 20% remaining for 2045+ (some of it from a coordinated AI Pause, which I think is likely to last if at all credibly established). If the 5 GW AI training systems don’t get built in 2028-2029, they are still likely to get built a bit later, so this essentially doesn’t influence predictions outside the 2029-2033 window, some probability within it merely gets pushed a bit towards the future.
So this gives a median of about 2034. Once AGI is still not working in the early 2030s even with more time for schlep, probability at that level of compute starts going down, so 2030s are front-loaded in probability even though compute is not scaling faster in the early 2030s than later.
But it isn’t August or September yet. Maybe someone will end up actually creating capable agents. In addition, the amount of operations used for creating Grok 4 was estimated as 4e27--6e27, which seems to align with the forecast. The research boost rate by Grok 4 or a potentially tweaked model wasn’t estimated. Maybe Grok 4 or an AI released in August will boost research speed?
It was indicated in the opening slide of Grok 4 release livestream that Grok 4 was pretrained with the same amount of compute as Grok 3, which in turn was pretrained on 100K H100s, so probably 3e26 FLOPs (40% utilization for 3 months with 1e15 FLOP/s per chip). RLVR has a 3x-4x lower compute utilization than pretraining, so if we are insisting on counting RLVR in FLOPs, then 3 months of RLVR might be 9e25 FLOPs, for the total of 4e26 FLOPs.
Stargate Abilene will be 400K chips in GB200 NVL72 racks in 2026, which is 10x more FLOP/s than 100K H100s. So it’ll be able to train 4e27-8e27 FLOPs models (pretraining and RLVR, in 3+3 months), and it might be early 2027 when they are fully trained. (Google is likely to remain inscrutable in their training compute usage, though Meta might also catch up by then.)
(I do realize it’s probably some sort of typo, either yours or in your unnamed source. But 10x is almost 2 years of even the current fast funding-fueled scaling, that’s not a small difference.)
We’ve been going on back and forth on this a bit—it seems like your model suggests AGI in 2027 is pretty unlikely?
That is, we see the first generation of massively scaled RLVR around 2026/2027. So it kind of has to work out of the box for AGI to arrive that quickly?
I suppose this is just speculation though. Maybe it’s useful enough that the next generation is somehow much, much faster to arrive?
By 2027, we’ll also have 10x scaled-up pretraining compared to current models (trained on 2024 compute). And correspondingly scaled RLVR, with many diverse tool-using environments that are not just about math and coding contest style problems. If we go 10x lower than current pretraining, we get original GPT-4 from Mar 2023, which is significantly worse than the current models. So with 10x higher pretraining than current models, the models of 2027 might make significantly better use of RLVR training than the current models can.
Also, 2 years might be enough time to get some sort of test-time training capability started, either with novel or currently-secret methods, or by RLVRing models to autonomously do post-training on variants of themselves to make them better at particular sources of tasks during narrow deployment. Apparently Sutskever’s SSI is rumored to be working on the problem (at 39:25 in the podcast), and overall this seems like the most glaring currently-absent faculty. (Once it’s implemented, something else might end up a similarly obvious missing piece.)
I’d give it 10% (for 2025-2027). From my impression of the current capabilities and the effect of scaling so far, the remaining 2 OOMs of compute seem like a 30% probability of getting there (by about 2030), with a third of it in the first 10x of the remaining scaling, that is 10% with 2026 compute (for 2027 models). After 2029, scaling slows down to a crawl (relatively speaking), so maybe another 50% for the 1000x of scaling in 2030-2045 when there’ll also be time for any useful schlep, with 20% remaining for 2045+ (some of it from a coordinated AI Pause, which I think is likely to last if at all credibly established). If the 5 GW AI training systems don’t get built in 2028-2029, they are still likely to get built a bit later, so this essentially doesn’t influence predictions outside the 2029-2033 window, some probability within it merely gets pushed a bit towards the future.
So this gives a median of about 2034. Once AGI is still not working in the early 2030s even with more time for schlep, probability at that level of compute starts going down, so 2030s are front-loaded in probability even though compute is not scaling faster in the early 2030s than later.
The section I shared is about mid 2025. I think August-September is late 2025.
Early: January, February, March, April
Mid: May, June, July, August
Late: September, October, November, December
Okay yes but this thread of discussion has gone long enough now I think—we basically agree up to a month.