More compute → people run more experiments → higher chance to discover better algorithms
More compute → people run larger experiments → higher chance to discover well scaling + high initial investment algorithms
It should be sufficiently counterbalanced by the growing cost per unit of compute for it to fail provide such increase, but I don’t think this is the case irl.
It is true that I neglected a relationship whereby compute can produce improvements in software. But here too, the exponential vs superexponential distinction matters. Under most reasonable functions that map to , doubling either doubles or less-than-doubles . This means that can’t manufacture super-exponential growth in , only an elevated exponential growth rate.
If one assumes (reasonably, IMHO) that the probability of SIE is monotonic in E_t, then “an elevated growth rate” trivially increases it. It doesn’t increase it super-exponentially, no… it’s merely an exponential increase in the probability of SIE. But exponentials get big suddenly, so I’m not sure why you’re objecting to this.
Under Eth and Davidson’s model – by far the most common and reasonable approach to modelling the SIE – a necessary condition for an SIE is that the growth rate of is itself growing. So “the probability of SIE is monotonic in ” is not true, or rather it’s only weakly monotonic.
What is your definition of an SIE? I’m fairly confident it involves an acceleration that is different from exponential growth in effective compute. Because exponential growth in effective compute has been true for the past decade, but nobody would classify the past decade as an SIE.
SIE = given some initial amount of compute and some level of capabilities, it enters regime where capabilities give it more capabilities, in short amount of time, without increasing the physical amount of compute. It achieves extremely high such levels of capability through this loop.
Growth here might not be actually super exponential, just extremely fast, e.g. in one day from subhuman to extremely superhuman in all domains, beating humans and humanity by extremely wide margin. Breaking previous capability growth trend with nearly vertical line. I would call this SIE.
Now, would it be more likely to happen with 1 P5 Pentium computer, or with 1000 H100s?
More compute → people run more experiments → higher chance to discover better algorithms → those algorithms make more compute directly via factories or optimize compute consumption efficiency → more compute
And anyway, “software intelligence explosion” is a feedback loop when intelligence gets you more intelligence.
It’s very obvious that more compute is conductive to the chances of you entering it. Either by providing more experiments or by allowing this feedback loop to be more inefficient and therefore easily reachable.
The expanded loop you’re describing is a chip production feedback loop, a different type of intelligence explosion. It’s possible, but there is AFAICT no evidence of a chip acceleration from AI, and even if it could happen, it is distinct from an SIE because the feedback loop takes place over a much longer time period. When people talk about an SIE, they’re concerned because it could happen quickly, at the speed of software. “Compute makes an SIE more likely” is not true for any reasonable definition of SIE,
And anyway, “software intelligence explosion” is a feedback loop when intelligence gets you more intelligence.
This is not true! Lots of things in the economy already meet this criterion. When Apple makes a Mac, they can use those Macs to speed up the process of creating future Macs. A power plant can use the electricity it generates to turn on the lights and make its workers more productive. “The output of a process feeds back in as an input” is a totally mundane condition that has nothing to do with an intelligence explosion.
What distinguishes an SIE is that intelligence alone gets you more intelligence. This leads to the “explosion” part, which like I said is underpinned by super exponential growth. That’s why my argument is about the mechanics of super exponential growth. If “intelligence leads to more intelligence” was capped at exponential growth, it would just be a business as usual scenario since that’s what we have now.
More compute → people run more experiments → higher chance to discover better algorithms
More compute → people run larger experiments → higher chance to discover well scaling + high initial investment algorithms
It should be sufficiently counterbalanced by the growing cost per unit of compute for it to fail provide such increase, but I don’t think this is the case irl.
It is true that I neglected a
That’s the crux, no?
If one assumes (reasonably, IMHO) that the probability of SIE is monotonic in E_t, then “an elevated growth rate” trivially increases it. It doesn’t increase it super-exponentially, no… it’s merely an exponential increase in the probability of SIE. But exponentials get big suddenly, so I’m not sure why you’re objecting to this.
Under Eth and Davidson’s model – by far the most common and reasonable approach to modelling the SIE – a necessary condition for an SIE is that the growth rate of
What is your definition of an SIE? I’m fairly confident it involves an acceleration that is different from exponential growth in effective compute. Because exponential growth in effective compute has been true for the past decade, but nobody would classify the past decade as an SIE.
SIE = given some initial amount of compute and some level of capabilities, it enters regime where capabilities give it more capabilities, in short amount of time, without increasing the physical amount of compute. It achieves extremely high such levels of capability through this loop.
Growth here might not be actually super exponential, just extremely fast, e.g. in one day from subhuman to extremely superhuman in all domains, beating humans and humanity by extremely wide margin. Breaking previous capability growth trend with nearly vertical line. I would call this SIE.
Now, would it be more likely to happen with 1 P5 Pentium computer, or with 1000 H100s?
How about
More compute → people run more experiments → higher chance to discover better algorithms → those algorithms make more compute directly via factories or optimize compute consumption efficiency → more compute
And anyway, “software intelligence explosion” is a feedback loop when intelligence gets you more intelligence.
It’s very obvious that more compute is conductive to the chances of you entering it. Either by providing more experiments or by allowing this feedback loop to be more inefficient and therefore easily reachable.
The expanded loop you’re describing is a chip production feedback loop, a different type of intelligence explosion. It’s possible, but there is AFAICT no evidence of a chip acceleration from AI, and even if it could happen, it is distinct from an SIE because the feedback loop takes place over a much longer time period. When people talk about an SIE, they’re concerned because it could happen quickly, at the speed of software. “Compute makes an SIE more likely” is not true for any reasonable definition of SIE,
This is not true! Lots of things in the economy already meet this criterion. When Apple makes a Mac, they can use those Macs to speed up the process of creating future Macs. A power plant can use the electricity it generates to turn on the lights and make its workers more productive. “The output of a process feeds back in as an input” is a totally mundane condition that has nothing to do with an intelligence explosion.
What distinguishes an SIE is that intelligence alone gets you more intelligence. This leads to the “explosion” part, which like I said is underpinned by super exponential growth. That’s why my argument is about the mechanics of super exponential growth. If “intelligence leads to more intelligence” was capped at exponential growth, it would just be a business as usual scenario since that’s what we have now.