In this post, Abram Demski argues that existing AI systems are already “AGI”. They are clearly general in a way previous generations of AI were not, and claiming that they are still not AGI smells of moving the goalposts.
Abram also helpfully edited the post to summarize and address some of the discussion in the comments. The commenters argued, and Abram largely agreed, that there are still important abilities that modern AI lacks. However, there is still the question of whether that should disqualify it from the moniker “AGI”, or maybe we need new terminology.
I tend to agree with Abram that there’s a sense in which modern AI is already “AGI”, and also agree with the commenters that there might be something important missing. To put the latter in my own words: I think that there is some natural property in computational-system-space s.t.
The prospect of AI with this property is the key reason to be worried about X-risk from unaligned AI.
Humans, or at least some humans, or at least humanity as a collective, has at least a little of this property, and this is what enabled humanity to become a technological civilization.
To handwave in the direction of that property, I would say “the ability to effectively and continuously acquire deep knowledge and exploit this knowledge to construct and execute goal-directed plans over long lifetimes and consequence horizons”.
It is IMO unclear whether modern AI are better thought of as having a positive but subhuman amount of this property, or as lacking it entirely (i.e. lacking some algorithmic component necessary for it). This question is hard to answer from our understanding of the algorithms, because foundation models “steal” some human cognitive algorithms in opaque ways, and we don’t even understand deep learning itself. Clearly, a civilization comprised of modern AI and no humans would not survive (not to mention progress), even if equipped with excellent robotic bodies. But, the latter might be just a “coincidental” fact about how harsh our specific universe is.
Be the case as it may, I think that the argument for more fine-grained terminology is strong. We can concede that modern AI is AGI, and have a new term for the thing modern AI might-not-yet-be. Maybe AGA: “Aritificial General Agent”?
In this post, Abram Demski argues that existing AI systems are already “AGI”. They are clearly general in a way previous generations of AI were not, and claiming that they are still not AGI smells of moving the goalposts.
Abram also helpfully edited the post to summarize and address some of the discussion in the comments. The commenters argued, and Abram largely agreed, that there are still important abilities that modern AI lacks. However, there is still the question of whether that should disqualify it from the moniker “AGI”, or maybe we need new terminology.
I tend to agree with Abram that there’s a sense in which modern AI is already “AGI”, and also agree with the commenters that there might be something important missing. To put the latter in my own words: I think that there is some natural property in computational-system-space s.t.
The prospect of AI with this property is the key reason to be worried about X-risk from unaligned AI.
Humans, or at least some humans, or at least humanity as a collective, has at least a little of this property, and this is what enabled humanity to become a technological civilization.
To handwave in the direction of that property, I would say “the ability to effectively and continuously acquire deep knowledge and exploit this knowledge to construct and execute goal-directed plans over long lifetimes and consequence horizons”.
It is IMO unclear whether modern AI are better thought of as having a positive but subhuman amount of this property, or as lacking it entirely (i.e. lacking some algorithmic component necessary for it). This question is hard to answer from our understanding of the algorithms, because foundation models “steal” some human cognitive algorithms in opaque ways, and we don’t even understand deep learning itself. Clearly, a civilization comprised of modern AI and no humans would not survive (not to mention progress), even if equipped with excellent robotic bodies. But, the latter might be just a “coincidental” fact about how harsh our specific universe is.
Be the case as it may, I think that the argument for more fine-grained terminology is strong. We can concede that modern AI is AGI, and have a new term for the thing modern AI might-not-yet-be. Maybe AGA: “Aritificial General Agent”?