All of these things are possible, but it’s not clear to me that they’re likely, at least in the early stages of AGI. In other words: once we have significantly-superhuman AGI, then agreed, all sorts of crazy things may become possible. But first we have to somehow achieve superhuman AGI. One of the things I’m trying to do in this post is explore the path that gets us to superhuman AGI in the first place. That path, by definition, can’t rely on anything that requires superhuman capabilities.
If I understand correctly, you’re envisioning that we will be able to construct AGIs that have human-level capability, and far greater than human speed, in order to bootstrap superhuman AGI? What makes you confident that this speed advantage will exist early on? Current leading-edge models like GPT-4 are not drastically faster than human beings, and presumably until we get to human-level AGI we’ll be spending most of our algorithmic improvements and increases in FLOP budget on increased capabilities, rather than performance. In fact, it’s quite possible that we’ll have to (temporarily) accept reduced speed in order to achieve human-level performance; for instance, by introducing tree search into the thought process (tree-of-thought prompting, heuristic search techniques in Yann LeCun’s “A Path Towards Autonomous Machine Intelligence”, etc.).
Once we achieve human-level, human-speed AGI, then yes, further algorithm or FLOPs improvements could be spent on speed; this comes back to the basic question of how whether the cognitive effort required for further progress increases more or less rapidly than the extent to which progress (and/or increased budgets) enables increased cognitive effort, i.e. does the self-improvement feedback loop converge or diverge. Are you proposing that it definitely diverges? What points you in that direction?
I would also caution against being highly confident that AGI will automatically be some sort of ideal omnimath. Such ability would require more than merely assimilating all of human knowledge and abilities; it would require knowing exactly which sub-specialties to draw on in any given moment. Otherwise, the AI would risk drowning in millions of superfluous connections to its every thought. Some examples of human genius might in part depend on a particular individual just happening to have precisely the right combination of knowledge, without a lot of other superfluous considerations to distract them.
Also, is it obvious that a single early-human-level AI could be trained with deep mastery of every field of human knowledge? Biological-anchor analysis aims to project when we can create a human-level AI, and humans are not omnimaths. Deep expertise across every subspeciality might easily require many more parameters than the number of synapses in the human brain. Many things look simple until you try to implement them; I touch on this in a recent blog post, The AI Progress Paradox, but you just need to look at the history of self-driving cars (or photorealistic CGI, or many other examples) to see how things that seem simple in principle can require many rounds of iteration to fully achieve in practice.
If I understand correctly, you’re envisioning that we will be able to construct AGIs that have human-level capability, and far greater than human speed, in order to bootstrap superhuman AGI? What makes you confident that this speed advantage will exist early on?
What I’m trying to say is that even at human speed, being able to mix-and-match human-level capabilities at will, in arbitrary combinations, not an ideal omnimath but in larger numbers than a single human can accumulate, is already a superhuman ability and one I expect AGI to trivially possess. Then on top of that you get, for free, things like being able to coordinate multiple instances of a single entity that don’t have their own other agendas, and that never lose focus or get tired.
Since you did mention genius coming from “precisely the right combination of knowledge, without a lot of other superfluous considerations to distract them,” I have to ask… doesn’t AGI seem perfectly positioned to be just that, for any combination of knowledge you can train it on?
I also don’t find the biological anchors argument convincing, for somewhat the same reason: an AI doesn’t need all of the superfluous knowledge a human has. Some of it, yes, but not all of it. To put it another way, in terms of data and parameters, how much knowledge of physics does a physicist actually have after a long career? A basic world model like all humans acquire in childhood, plus a few hundred books, a few thousand hours of lectures, and maybe 40k hours of sensory data acquired and thinking completed on-the-job?
And you’re right, I agree an early AGI won’t be an omnimath, but I think polymath is very much within reach.
All of these things are possible, but it’s not clear to me that they’re likely, at least in the early stages of AGI. In other words: once we have significantly-superhuman AGI, then agreed, all sorts of crazy things may become possible. But first we have to somehow achieve superhuman AGI. One of the things I’m trying to do in this post is explore the path that gets us to superhuman AGI in the first place. That path, by definition, can’t rely on anything that requires superhuman capabilities.
If I understand correctly, you’re envisioning that we will be able to construct AGIs that have human-level capability, and far greater than human speed, in order to bootstrap superhuman AGI? What makes you confident that this speed advantage will exist early on? Current leading-edge models like GPT-4 are not drastically faster than human beings, and presumably until we get to human-level AGI we’ll be spending most of our algorithmic improvements and increases in FLOP budget on increased capabilities, rather than performance. In fact, it’s quite possible that we’ll have to (temporarily) accept reduced speed in order to achieve human-level performance; for instance, by introducing tree search into the thought process (tree-of-thought prompting, heuristic search techniques in Yann LeCun’s “A Path Towards Autonomous Machine Intelligence”, etc.).
Once we achieve human-level, human-speed AGI, then yes, further algorithm or FLOPs improvements could be spent on speed; this comes back to the basic question of how whether the cognitive effort required for further progress increases more or less rapidly than the extent to which progress (and/or increased budgets) enables increased cognitive effort, i.e. does the self-improvement feedback loop converge or diverge. Are you proposing that it definitely diverges? What points you in that direction?
I would also caution against being highly confident that AGI will automatically be some sort of ideal omnimath. Such ability would require more than merely assimilating all of human knowledge and abilities; it would require knowing exactly which sub-specialties to draw on in any given moment. Otherwise, the AI would risk drowning in millions of superfluous connections to its every thought. Some examples of human genius might in part depend on a particular individual just happening to have precisely the right combination of knowledge, without a lot of other superfluous considerations to distract them.
Also, is it obvious that a single early-human-level AI could be trained with deep mastery of every field of human knowledge? Biological-anchor analysis aims to project when we can create a human-level AI, and humans are not omnimaths. Deep expertise across every subspeciality might easily require many more parameters than the number of synapses in the human brain. Many things look simple until you try to implement them; I touch on this in a recent blog post, The AI Progress Paradox, but you just need to look at the history of self-driving cars (or photorealistic CGI, or many other examples) to see how things that seem simple in principle can require many rounds of iteration to fully achieve in practice.
What I’m trying to say is that even at human speed, being able to mix-and-match human-level capabilities at will, in arbitrary combinations, not an ideal omnimath but in larger numbers than a single human can accumulate, is already a superhuman ability and one I expect AGI to trivially possess. Then on top of that you get, for free, things like being able to coordinate multiple instances of a single entity that don’t have their own other agendas, and that never lose focus or get tired.
Since you did mention genius coming from “precisely the right combination of knowledge, without a lot of other superfluous considerations to distract them,” I have to ask… doesn’t AGI seem perfectly positioned to be just that, for any combination of knowledge you can train it on?
I also don’t find the biological anchors argument convincing, for somewhat the same reason: an AI doesn’t need all of the superfluous knowledge a human has. Some of it, yes, but not all of it. To put it another way, in terms of data and parameters, how much knowledge of physics does a physicist actually have after a long career? A basic world model like all humans acquire in childhood, plus a few hundred books, a few thousand hours of lectures, and maybe 40k hours of sensory data acquired and thinking completed on-the-job?
And you’re right, I agree an early AGI won’t be an omnimath, but I think polymath is very much within reach.