it’s notable that we expect ~1 month in not that long, like only ~3 years at the current rate
That’s only if the faster within-RLVR rate that has been holding during the last few months persists. On my current model, 1 month task lengths at 50% happen in 2030-2032, since compute (being the scarce input of scaling) slows down compared to today, and I don’t particularly believe in incremental algorithmic progress as it’s usually quantified, so it won’t be coming to the rescue.
Compared to the post I did on this 4 months ago, I have even lower expectations that the 5 GW training systems (for individual AI companies) will arrive on trend in 2028, they’ll probably get delayed to 2029-2031. And I think the recent RLVR acceleration of the pre-RLVR trend only pushes it forward a year without making it faster, the changed “trend” of the last few months is merely RLVR chip-hours catching up to pretraining chip-hours, which is already essentially over. Though there are still no GB200 NVL72 sized frontier models and probably no pretraining scale RLVR on GB200 NVL72s (which would get better compute utilization), so that might give the more recent “trend” another off-trend push first, perhaps as late as early 2026, but then it’s not yet a whole year ahead of the old trend.
That’s only if the faster within-RLVR rate that has been holding during the last few months persists. On my current model, 1 month task lengths at 50% happen in 2030-2032, since compute (being the scarce input of scaling) slows down compared to today, and I don’t particularly believe in incremental algorithmic progress as it’s usually quantified, so it won’t be coming to the rescue.
Compared to the post I did on this 4 months ago, I have even lower expectations that the 5 GW training systems (for individual AI companies) will arrive on trend in 2028, they’ll probably get delayed to 2029-2031. And I think the recent RLVR acceleration of the pre-RLVR trend only pushes it forward a year without making it faster, the changed “trend” of the last few months is merely RLVR chip-hours catching up to pretraining chip-hours, which is already essentially over. Though there are still no GB200 NVL72 sized frontier models and probably no pretraining scale RLVR on GB200 NVL72s (which would get better compute utilization), so that might give the more recent “trend” another off-trend push first, perhaps as late as early 2026, but then it’s not yet a whole year ahead of the old trend.