For most growth curves, the effect is important on reasonable time scales. Most of the time we’re trying to model something where growth will run out in years or decades (though perhaps to be replaced by another S-curve due to innovation).
For example, it should have been obvious to everyone that LLM scaling was on an S-curve, and that scale alone would run out of gains and we’d have to start looking for gains from sources other than scale. That many modelers before 2025 thought that LLM gains could go on forever from scaling alone is a failure of their models. Had they even tried to model when scaling would run out, they could have better anticipated when future growth in capabilities would have had to come from other sources. But most models I saw didn’t even attempt this.
I’m surprised at how hard it is for me to think of counterexamples.
I thought surely whale populations due to the slow generation time, but it looks like humpback whale populations have already recovered from whaling, and blue whales will get there before long.
Thinking again—in my baseball example, gravity is pulling the ball into the domain of applicability of the constant acceleration model.
Maybe what’s special about the exponential growth model is it implies escape from its own domain of applicability, in time that grows slowly (logarithmically) with the threshold.
For most growth curves, the effect is important on reasonable time scales. Most of the time we’re trying to model something where growth will run out in years or decades (though perhaps to be replaced by another S-curve due to innovation).
For example, it should have been obvious to everyone that LLM scaling was on an S-curve, and that scale alone would run out of gains and we’d have to start looking for gains from sources other than scale. That many modelers before 2025 thought that LLM gains could go on forever from scaling alone is a failure of their models. Had they even tried to model when scaling would run out, they could have better anticipated when future growth in capabilities would have had to come from other sources. But most models I saw didn’t even attempt this.
I’m surprised at how hard it is for me to think of counterexamples.
I thought surely whale populations due to the slow generation time, but it looks like humpback whale populations have already recovered from whaling, and blue whales will get there before long.
Thinking again—in my baseball example, gravity is pulling the ball into the domain of applicability of the constant acceleration model.
Maybe what’s special about the exponential growth model is it implies escape from its own domain of applicability, in time that grows slowly (logarithmically) with the threshold.