Twenty Years from RSI to Takeoff: Slow Learning, Scaling Slowdown, Industrial Explosion
Industrial explosion is what will make the next-model building loops (and thus learning) with LLMs 1000x faster by about 2050, if indeed the slow-learning prosaic RSI becomes AGI before the big compute buildout slowdown of 2032+ that is already starting. This puts an upper bound on how long it takes to invent ASI that sets off software-only singularity, implementing efficient online learning and fixing all the other hobblings of the likely near-future AGI technology (LLMs/pretraining/RL). The invention of ASI in that sense is still possible at any time (and very quickly scales, given all the compute), but the likely initial state of slow-learning AGIs of 2028-2032 doesn’t seem to give them a significant advantage over humanity in getting there faster. And so it doesn’t seem too unlikely that nothing substantively new gets invented until 2040-2050, when the LLM/RL AGIs start accelerating because of the industrial explosion they set off.
Fast Reasoning, Slow Learning
The current methods are likely to enable automated general learning (thus AGI) very soon, using automated creation of RL tasks/environments/graders filling the visible gaps in model capability for the topics and situations that happen to be borderline unfamiliar for that model, followed by automated next-model building. This teaches LLMs deep skills, but operates at the speed of next-model building (weeks to months for one iteration of advancing the deep skill frontier) rather than at the speed of next-token generation (100-1000x the human speed). Humans are not the key bottleneck to the speed of next-model building loops, there’s still a lot of waiting for the compute to do its thing in training, so achieving prosaic RSI by teaching the near-future LLMs all the skills necessary to perform it automatically doesn’t make it go too fast. Using smaller LLMs to make everything faster doesn’t work because the current frontier LLMs are probably borderline insufficient for learning the prosaic RSI skills that automate the next-model building loop. The LLMs of 2028-2031 (that are very likely sufficient) will be even bigger, though the cost of training or running them is not as bad as “quadrillion total params” sounds. This cost can’t be circumvented by using different hardware that makes LLM inference much faster, because different hardware doesn’t reduce the necessary number of FLOPs, which are not terribly wasted even in RL training and inference that involve bandwidth bound decode. Fundamentally, cost is the amount of compute, and the only thing that overcomes it is the scale of the global buildout.
Compute Slowdown, Industrial Explosion
Since LLMs become ready to close the next-model building loop (that enables AGI) just as the human industry runs out of various kinds of fuel for quickly increasing the scale of the compute buildout, there is no opportunity for another near-term 1000x increase in the available compute (and thus the speed of next-model building loops), the way compute was increasing in 2022-2027, and the way it’ll keep increasing (a bit slower) in 2028-2032 until the pace of decommissioning old compute somewhat catches up to the 2028+ pace of producing new compute, set by factors like availability of EUV machines and skilled human labor. Thus the big compute slowdown of 2032+, stronger than the end of the current exponential scaling of compute by 2028+.
Without paradigm-breaking algorithmic innovations, the slow-learning AGIs can’t quickly invent such innovations, and so the more predictable component of the pace of progress is set by the pace of the compute buildout. But also, the AGIs (likely available since 2028-2032) make the industrial explosion of robot-building robots a predictable medium-term development. It probably doesn’t start right away, since the AGIs are not much faster than humanity at taking care of all the novel engineering challenges (requiring many next-model building loops to get good), and before it goes into full swing humanity still needs to handle the industrial side of things (at the human pace).
The automotive industry and the compute buildout acceleration of 2022-2027 seem like good anchors for how this might unfold. The process starts once AGIs unlock an outsized demand for robots (by making them very useful for everything), and the industry starts reshaping itself to increase the supply as fast as it can, similarly to the consequences of the ChatGPT moment. Robot production exhausts the industrial capacity of the supply chains within 3-5 years (similarly to how it took 5-6 years to reshape compute production). At that point, the scale of the robot supply (anchored to the current automotive industry) approaches a significant portion of human labor, so the process continues right past the limits of human industry without another big slowdown. If the doubling time of the robot-building industry (autonomously operated by AGIs using the existing robots) is around 1 year, then 10 years of this process increase the industrial capacity about 1000x. The countdown should probably start from the end of the 3-5 year period of industrial conversion, when the robot industry first matches a sufficient fraction of the human industry to also start producing as much compute (together with all the other precursors), and the slow-learning AGIs probably need the time for the next-model building cycles to figure out how to automate everything.
Prosaic Timeline to Takeoff
The timeline starts with prosaic RSI in 2028-2032. The resulting slow-learning AGIs first make robots very useful generally within 2-3 years, in 2030-2034, setting off the industrial conversion of human industry towards robot production. This lasts another 3-5 years, and by 2034-2039 the automated robot-building industry operated by robots and AGIs first matches the human industry’s capacity in terms of the compute buildout it can support. If this industry can quickly reach a doubling time of 1 year, it can 1000x the compute buildout by 2044-2050. At that point, the next-model building cycles take 1000x less time, and so the unpredictable paradigm-breaking algorithmic innovations necessary to set off a software-only singularity happen on the scale of months instead of centuries, and would’ve happened at some point earlier than that, probably by 2040-2045, but certainly by 2050. This is the upper bound on the timing of feasibility of superintelligence, which mostly assumes just the current paradigm (extremely hobbled in its efficacy at superhuman invention) rather than any particular future breakthroughs.
This post is surprising to me. In the past few years, models have been getting smarter so fast that superintelligence seems very close, probably no further than 2030. That’s without looking at spending at all (in dollars, flops or whatever). You seem to be saying, based on spending, that it’s actually much further out. How can that be true?
“it doesn’t seem too unlikely that nothing substantively new gets invented until 2040-2050”
This seems like it would be the longest single drought in major industrial / civilizational advances since we invented flight, at the vey latest, and the slowest in computational methods and advances since before the vacuum tube—why would you expect everything to slow down so much?
A key part of the answer is that algorithmic progress is scale-dependent to an extent that is often not realized, and >90% of algorithmic progress is basically downstream of scaling compute, so any slow down in the scaling of compute automatically slows down all other progress as a side effect:
Here’s a useful article below:
Notes on Implications of Scale-Dependent Algorithms
...but then this is an argument that more compute won’t be available, and the chip roadmap for NVIDIA goes through the Feynman Architecture in 2028, then panel level packaging and HBM5, along with increased production volumes—so it seems implausible we’d see a compute availability slowdown?
Also, the ‘algorithmic progress is actually compute progress’ argument is far older and more general than the version they present for LLMs, and is convincing to some extent, but also weaker than I think it appears if you look at the data.
The slowdown is relative to the 2022-2027 exponential. After 2028-2029, there will be increasingly serious problems with increasing incremental volume (the amount of the new buildout per year). Then there are a few years of accumulation at a slowly increasing incremental volume, and around 2032 decomissioned (or no-longer-relevant) old compute starts catching up with the new compute (see the plot under the words “how silicon capacity holds back AI chip deployment”).
Thanks; that makes sense, but I think it doesn’t make the case I think you’re expecting. The claim linked in that article is that TSMC won’t be able to get cleanroom space in 2026/2027; unless they are blind, they’ll avoid repeating that mistake for 2028/2029 - https://newsletter.semianalysis.com/p/the-great-ai-silicon-shortage But even if all of that happens, the new chips are faster, and will be made available; the increase of NVIDIA’s chips just won’t be quite as exponentially large as historically.
But even then, it’s not like Google’s TPUs and others are so far behind that a multi-year fumble wouldn’t allow other firms to catch up, and China is certainly going all in. So even if the analysis is correct, it won’t eliminate the exponential trend.
Chips will get faster, but chips are made out of logic dies, and transistors no longer shrink that much. So FLOP/s per GW won’t get a lot better, and silicon capacity is binding for FLOP/s (which is more about EUV machines than TSMC fabs). Future process nodes consume more EUV cycles per wafer, so global incremental FLOP/s manufactured per year don’t obviously grow (notably) faster than the number of available EUV machines.
I’m not saying the lack of the relevant paradigm-breaking innovations is likely, merely that it’s “not too unlikely”, so it’s still worth considering. It did in some sense take more than 60 years to invent transformers. This post explores an upper bound on how long it takes to set off a software-only singularity. It might still happen at any moment, as I mentioned in the post, and the current compute already seems sufficient to scale it well past a takeover.
I’d give 50% for takeoff happening by 2032 (before the compute slowdown that precedes industrial explosion), the substantive observation in the post is that it won’t be happening after 2060 without a ban/pause or another disruption, and this estimate doesn’t depend on how long it takes to do the basic research (because by 2045-2050 even the “slow-learning” LLM/RL AGIs will have completed centuries worth of research).
“It did in some sense take more than 60 years to invent transformers.”
I think this is wrong in a meaningful sense; transformers weren’t useful until we had enough compute. A bare-minimum transformer model would be tens of millions of parameters, and require tens of millions of FLOPs to do inference, and a large multiple of that to train. So it’s like saying we didn’t have Minecraft back then; true, but not useful; it couldn’t have been written until computers were fast enough and had good enough graphics.
Thank you for sharing! I got a kick out of the term “prosaic RSI” haha. I think all your timelines seem reasonable to me as an upper bound. The main place I’d differ is that I expect robot mass production to begin before AGI makes robots economically profitable. As we’ve seen with the compute buildout, investors and governments are perfectly willing to pour money and resources into an industry if they believe it will be strategically critical in the future. So I’d probably move that milestone back by a few years.
I don’t understand even this level of pessimism; robots are already profitable, and they have grown 5x in the past 20 years, to be a $50b industry, and the “coming” wave of investment already started; projections from “optimists” have it growing to $2.5tr by 2035, and they aren’t banking on ASI at all, just current methods working out increasingly well.
I presume you meant humanoid robots not industrial ones and not vacuum cleaners, right?
~85% of the humanoid robots are sold in China, and about 3⁄4 of all the demand in China is from the education and research sectors—including, notably, government-backed training centers which sell the data back to the manufacturers: https://www.ft.com/content/26735a23-315f-47ef-8cf2-6c6ea9713998
AI datacenters also existed before the ChatGPT moment, they just didn’t get to a trillion dollars of buildout per year until a few years after the unit-level profitability (at a large TAM) prompted the industrial conversion. Similarly, there are some robots now, and there will be more robots before they become very useful for everything, but getting to the kind of scale that can match human industry’s compute buildout as its side effect will still take a lot of time after the robots become profitable at the unit level (with the kind of somewhat-concretely-visible TAM that dwarfs the automotive industry).
It really seems like you’re assuming that investors won’t notice a widely predicted trend that starts materializing quickly enough to make money on it, which...?
I understand that there are some bottlenecks in robotics equipment, but they aren’t as fundamental as the UV lithography constrain for chips—and one of the key bottlenecks is chips, and we’re obviously seeing lots of money invested in building out that capacity already.
My claims aren’t that different from what you said in the other comment:
The timeline in this post puts parity with automotive industry at 2034-2039, which is currently at about 2-3 trillion dollars of revenue per year. I’m also explicitly not banking on ASI at all with the conservative assumptions of this post.