Oh, come on. That is straight-up not how simple continuous toy models of RSI work. Between a neutron multiplication factor of 0.999 and 1.001 there is a very huge gap in output behavior.
Nitpick: I think that particular analogy isn’t great.
For nuclear stuff, we have two state variables: amount of fissile material and current number of neutrons flying around. The amount of fissile material determines the “neutron multiplication factor”, but it is the number of neutrons that goes crazy, not fissile material. And the current number of neurons doesn’t matter for whether the pile will eventually go crazy or not.
But in the simplest toy models of RSI, we just have one variable: intelligence. We can’t change the “intelligence multiplication factor”, there’s just intelligence figuring out how to build more intelligence.
Maybe exothermic chemical reactions, like fire, is a better analogy. Either you have enough heat to create a self-sustaining reaction, or you don’t.
Nitpick: I think that particular analogy isn’t great.
For nuclear stuff, we have two state variables: amount of fissile material and current number of neutrons flying around. The amount of fissile material determines the “neutron multiplication factor”, but it is the number of neutrons that goes crazy, not fissile material. And the current number of neurons doesn’t matter for whether the pile will eventually go crazy or not.
But in the simplest toy models of RSI, we just have one variable: intelligence. We can’t change the “intelligence multiplication factor”, there’s just intelligence figuring out how to build more intelligence.
Maybe exothermic chemical reactions, like fire, is a better analogy. Either you have enough heat to create a self-sustaining reaction, or you don’t.