It is true that I neglected a relationship whereby compute can produce improvements in software. But here too, the exponential vs superexponential distinction matters. Under most reasonable functions that map to , doubling either doubles or less-than-doubles . This means that can’t manufacture super-exponential growth in , only an elevated exponential growth rate.
If one assumes (reasonably, IMHO) that the probability of SIE is monotonic in E_t, then “an elevated growth rate” trivially increases it. It doesn’t increase it super-exponentially, no… it’s merely an exponential increase in the probability of SIE. But exponentials get big suddenly, so I’m not sure why you’re objecting to this.
Under Eth and Davidson’s model – by far the most common and reasonable approach to modelling the SIE – a necessary condition for an SIE is that the growth rate of is itself growing. So “the probability of SIE is monotonic in ” is not true, or rather it’s only weakly monotonic.
What is your definition of an SIE? I’m fairly confident it involves an acceleration that is different from exponential growth in effective compute. Because exponential growth in effective compute has been true for the past decade, but nobody would classify the past decade as an SIE.
More compute → people run more experiments → higher chance to discover better algorithms → those algorithms make more compute directly via factories or optimize compute consumption efficiency → more compute
And anyway, “software intelligence explosion” is a feedback loop when intelligence gets you more intelligence.
It’s very obvious that more compute is conductive to the chances of you entering it. Either by providing more experiments or by allowing this feedback loop to be more inefficient and therefore easily reachable.
The expanded loop you’re describing is a chip production feedback loop, a different type of intelligence explosion. It’s possible, but there is AFAICT no evidence of a chip acceleration from AI, and even if it could happen, it is distinct from an SIE because the feedback loop takes place over a much longer time period. When people talk about an SIE, they’re concerned because it could happen quickly, at the speed of software. “Compute makes an SIE more likely” is not true for any reasonable definition of SIE,
And anyway, “software intelligence explosion” is a feedback loop when intelligence gets you more intelligence.
This is not true! Lots of things in the economy already meet this criterion. When Apple makes a Mac, they can use those Macs to speed up the process of creating future Macs. A power plant can use the electricity it generates to turn on the lights and make its workers more productive. “The output of a process feeds back in as an input” is a totally mundane condition that has nothing to do with an intelligence explosion.
What distinguishes an SIE is that intelligence alone gets you more intelligence. This leads to the “explosion” part, which like I said is underpinned by super exponential growth. That’s why my argument is about the mechanics of super exponential growth. If “intelligence leads to more intelligence” was capped at exponential growth, it would just be a business as usual scenario since that’s what we have now.
It is true that I neglected a
That’s the crux, no?
If one assumes (reasonably, IMHO) that the probability of SIE is monotonic in E_t, then “an elevated growth rate” trivially increases it. It doesn’t increase it super-exponentially, no… it’s merely an exponential increase in the probability of SIE. But exponentials get big suddenly, so I’m not sure why you’re objecting to this.
Under Eth and Davidson’s model – by far the most common and reasonable approach to modelling the SIE – a necessary condition for an SIE is that the growth rate of
What is your definition of an SIE? I’m fairly confident it involves an acceleration that is different from exponential growth in effective compute. Because exponential growth in effective compute has been true for the past decade, but nobody would classify the past decade as an SIE.
How about
More compute → people run more experiments → higher chance to discover better algorithms → those algorithms make more compute directly via factories or optimize compute consumption efficiency → more compute
And anyway, “software intelligence explosion” is a feedback loop when intelligence gets you more intelligence.
It’s very obvious that more compute is conductive to the chances of you entering it. Either by providing more experiments or by allowing this feedback loop to be more inefficient and therefore easily reachable.
The expanded loop you’re describing is a chip production feedback loop, a different type of intelligence explosion. It’s possible, but there is AFAICT no evidence of a chip acceleration from AI, and even if it could happen, it is distinct from an SIE because the feedback loop takes place over a much longer time period. When people talk about an SIE, they’re concerned because it could happen quickly, at the speed of software. “Compute makes an SIE more likely” is not true for any reasonable definition of SIE,
This is not true! Lots of things in the economy already meet this criterion. When Apple makes a Mac, they can use those Macs to speed up the process of creating future Macs. A power plant can use the electricity it generates to turn on the lights and make its workers more productive. “The output of a process feeds back in as an input” is a totally mundane condition that has nothing to do with an intelligence explosion.
What distinguishes an SIE is that intelligence alone gets you more intelligence. This leads to the “explosion” part, which like I said is underpinned by super exponential growth. That’s why my argument is about the mechanics of super exponential growth. If “intelligence leads to more intelligence” was capped at exponential growth, it would just be a business as usual scenario since that’s what we have now.