Because each was a factor of two increase, and it is a general rule that each doubling tends to bring about the same increase in capability.
I found this an interesting rule, and thought that after the examples you would establish some firmer theoretical basis for why it might work the way it does. But you didn’t do that, and instead jumped to talking about AGI. It feels like you’re trying to apply a rule before having established how and why it works, which raises “possible reasoning by surface analogy” warning bells in my head. The tone in your post is a lot more confident than it should be.
Fair enough; it’s true that I don’t have a rigorous mathematical model, nor do I expect to have one anytime soon. My current best sketch at an explanation is that it’s a combination of:
P != NP, i.e. the familiar exponential difficulty of finding solutions given only a way to evaluate them—I think this is the part that contributes the overall exponential shape.
The general version of Amdahl’s law, if subproblem X was only e.g. 10% of the overall job, then no improvement in X can by itself give more than a 10% overall improvement—I think this is the part that makes it so robust; as I mentioned in the subthread about algorithmic improvements, even if you can break the curve of capability in a subproblem, it persists at the next level up.
I found this an interesting rule, and thought that after the examples you would establish some firmer theoretical basis for why it might work the way it does. But you didn’t do that, and instead jumped to talking about AGI. It feels like you’re trying to apply a rule before having established how and why it works, which raises “possible reasoning by surface analogy” warning bells in my head. The tone in your post is a lot more confident than it should be.
Fair enough; it’s true that I don’t have a rigorous mathematical model, nor do I expect to have one anytime soon. My current best sketch at an explanation is that it’s a combination of:
P != NP, i.e. the familiar exponential difficulty of finding solutions given only a way to evaluate them—I think this is the part that contributes the overall exponential shape.
The general version of Amdahl’s law, if subproblem X was only e.g. 10% of the overall job, then no improvement in X can by itself give more than a 10% overall improvement—I think this is the part that makes it so robust; as I mentioned in the subthread about algorithmic improvements, even if you can break the curve of capability in a subproblem, it persists at the next level up.