Compute growth doesn’t make a software intelligence explosion more likely

[Karthik Tadepalli]

This is a nitpick of a claim that I sometimes see in discussions of a software intelligence explosion. There is a careful analysis of the conditions under which an SIE will occur, assuming a fixed stock of compute. Then the argument concludes with “And in reality, compute is not fixed and is in fact growing exponentially, so an SIE is even more likely than this analysis suggests.”

This conclusion is false. Exponentially growing physical compute has no effect on the likelihood of an SIE. To see this, let

denote effective compute, where is physical compute and is algorithmic efficiency. Better algorithms allow a fixed stock of hardware to perform as much work as a larger stock of physical compute – the SIE channel. But what does the physical compute channel do?

Takings logs and differentiating gives the relationship between the growth rates of each of these quantities:

A necessary condition for an SIE is – the growth rate of effective compute is itself growing. ^[1] This means an SIE can happen only if

An exponentially growing compute stock means – but it also means is constant, so . As a result, exponential compute growth contributes nothing to the SIE. The necessary condition remains , in which physical compute doesn’t appear.

What’s going on here? Intuitively, an SIE requires super-exponential growth in , which requires either or to grow super-exponentially. The standard SIE analysis posits conditions for to grow super-exponentially, based on the feedback loop from self-improving AI. But if is growing super-exponentially, then will grow super-exponentially even if is fixed – so growth in is redundant. On the other hand, if only grows exponentially, exponential growth in can’t push it into super-exponential territory. The product of two exponentials is still exponential.

The only way for to contribute to an intelligence explosion is if it grows super-exponentially, in a chip-production/​chip technology feedback loop – but this is a much stronger condition than any required for an SIE.

Of course, growth in physical compute still raises the rate of AI progress – so it still shortens doubling times for any measure of AI capabilities. But it doesn’t move the needle on the specific question of how likely is a software-only intelligence explosion.

1. ↩︎

   This is not a sufficient condition – the growth rate also needs to be fast enough to outrun diminishing returns to research, which is the condition from Eth and Davidson (2025) – but its necessity is enough for our purposes.

[Canaletto]

More compute → people run more experiments → higher chance to discover better algorithms

More compute → people run larger experiments → higher chance to discover well scaling + high initial investment algorithms

It should be sufficiently counterbalanced by the growing cost per unit of compute for it to fail provide such increase, but I don’t think this is the case irl.

[Karthik Tadepalli]

It is true that I neglected a relationship whereby compute can produce improvements in software. But here too, the exponential vs superexponential distinction matters. Under most reasonable functions that map to , doubling either doubles or less-than-doubles . This means that can’t manufacture super-exponential growth in , only an elevated exponential growth rate.

[dan.parshall]

That’s the crux, no?

If one assumes (reasonably, IMHO) that the probability of SIE is monotonic in E_t, then “an elevated growth rate” trivially increases it. It doesn’t increase it super-exponentially, no… it’s merely an exponential increase in the probability of SIE. But exponentials get big suddenly, so I’m not sure why you’re objecting to this.

[Karthik Tadepalli]

Under Eth and Davidson’s model – by far the most common and reasonable approach to modelling the SIE – a necessary condition for an SIE is that the growth rate of is itself growing. So “the probability of SIE is monotonic in ” is not true, or rather it’s only weakly monotonic.

What is your definition of an SIE? I’m fairly confident it involves an acceleration that is different from exponential growth in effective compute. Because exponential growth in effective compute has been true for the past decade, but nobody would classify the past decade as an SIE.

[Canaletto]

SIE = given some initial amount of compute and some level of capabilities, it enters regime where capabilities give it more capabilities, in short amount of time, without increasing the physical amount of compute. It achieves extremely high such levels of capability through this loop.

Growth here might not be actually super exponential, just extremely fast, e.g. in one day from subhuman to extremely superhuman in all domains, beating humans and humanity by extremely wide margin. Breaking previous capability growth trend with nearly vertical line. I would call this SIE.

Now, would it be more likely to happen with 1 P5 Pentium computer, or with 1000 H100s?

[Canaletto]

How about

More compute → people run more experiments → higher chance to discover better algorithms → those algorithms make more compute directly via factories or optimize compute consumption efficiency → more compute

And anyway, “software intelligence explosion” is a feedback loop when intelligence gets you more intelligence.

It’s very obvious that more compute is conductive to the chances of you entering it. Either by providing more experiments or by allowing this feedback loop to be more inefficient and therefore easily reachable.

[Karthik Tadepalli]

The expanded loop you’re describing is a chip production feedback loop, a different type of intelligence explosion. It’s possible, but there is AFAICT no evidence of a chip acceleration from AI, and even if it could happen, it is distinct from an SIE because the feedback loop takes place over a much longer time period. When people talk about an SIE, they’re concerned because it could happen quickly, at the speed of software. “Compute makes an SIE more likely” is not true for any reasonable definition of SIE,

And anyway, “software intelligence explosion” is a feedback loop when intelligence gets you more intelligence.

This is not true! Lots of things in the economy already meet this criterion. When Apple makes a Mac, they can use those Macs to speed up the process of creating future Macs. A power plant can use the electricity it generates to turn on the lights and make its workers more productive. “The output of a process feeds back in as an input” is a totally mundane condition that has nothing to do with an intelligence explosion.

What distinguishes an SIE is that intelligence alone gets you more intelligence. This leads to the “explosion” part, which like I said is underpinned by super exponential growth. That’s why my argument is about the mechanics of super exponential growth. If “intelligence leads to more intelligence” was capped at exponential growth, it would just be a business as usual scenario since that’s what we have now.