I have an analogy that may help distinguish the two AI systems we are talking about. Those being: 1) the RETRO-type narrow language AIs we already possess, which are able to modify a cache of memory to aid in their operations, while their weights and biases are frozen, 2) a temporal, life-long learning AGI which is able to add new functionalities without limit, such that we cannot contain it, and it is guaranteed to overwhelm us.
That second, life-long learner is the ‘AGI domination-risk’ because it can add new capabilities without bounds. RETRO and its ilk are certainly not able to add capabilities without bounds, such that they are not a risk in the same ways (mostly mis-use).
The Analogy:
RETRO and other memory-cache equipped narrow AIs are like a piece of software that can have multiple global and local state variables. These state variables allow the software to act along numerous ‘paths’ within the totality of what its program can already do; yet these state variables do NOT allow the program to acquire new capabilities “at-will and without bounds”. You have to write a new piece of software for that. The risk is AGI writing more capabilities onto itself, not a “narrow AI switching between pre-existing settings.”
Do you have a rebuttal to my rebuttal, or any further critique? Yours is the same, singular response I’ve heard from the AI safety folks I’ve spoken to, in Berkeley, and none of you have had a follow-up when I point-out the distinction… Don’t I stand provisionally correct, until I stand corrected?
In practice the obvious way to construct an AI stack allows the AI itself to just be a product of another optimization engine. The model and it’s architecture are generated from a higher level system that allocates a design to satisfy some higher level constraint. So the AI itself is pretty mutable, it can’t add capabilities without limit because the capabilities must be in the pursuit of the system’s design heuristic written by humans, but humans aren’t needed to add capabilities.
In practice from a human perspective, a system that has some complex internal state variables can fail in many many ways. It’s quite unreliable inherently. It’s why in your own life you have seen many system failures—they almost all failed from complex internal state. It’s why your router fails, why a laptop fails, a game console fails to update, a car infotainment system fails to update, a system at the DMV or a school or hospital goes down, and so on.
I have an analogy that may help distinguish the two AI systems we are talking about. Those being: 1) the RETRO-type narrow language AIs we already possess, which are able to modify a cache of memory to aid in their operations, while their weights and biases are frozen, 2) a temporal, life-long learning AGI which is able to add new functionalities without limit, such that we cannot contain it, and it is guaranteed to overwhelm us.
That second, life-long learner is the ‘AGI domination-risk’ because it can add new capabilities without bounds. RETRO and its ilk are certainly not able to add capabilities without bounds, such that they are not a risk in the same ways (mostly mis-use).
The Analogy:
RETRO and other memory-cache equipped narrow AIs are like a piece of software that can have multiple global and local state variables. These state variables allow the software to act along numerous ‘paths’ within the totality of what its program can already do; yet these state variables do NOT allow the program to acquire new capabilities “at-will and without bounds”. You have to write a new piece of software for that. The risk is AGI writing more capabilities onto itself, not a “narrow AI switching between pre-existing settings.”
Do you have a rebuttal to my rebuttal, or any further critique? Yours is the same, singular response I’ve heard from the AI safety folks I’ve spoken to, in Berkeley, and none of you have had a follow-up when I point-out the distinction… Don’t I stand provisionally correct, until I stand corrected?
Couple of comments here.
In practice the obvious way to construct an AI stack allows the AI itself to just be a product of another optimization engine. The model and it’s architecture are generated from a higher level system that allocates a design to satisfy some higher level constraint. So the AI itself is pretty mutable, it can’t add capabilities without limit because the capabilities must be in the pursuit of the system’s design heuristic written by humans, but humans aren’t needed to add capabilities.
In practice from a human perspective, a system that has some complex internal state variables can fail in many many ways. It’s quite unreliable inherently. It’s why in your own life you have seen many system failures—they almost all failed from complex internal state. It’s why your router fails, why a laptop fails, a game console fails to update, a car infotainment system fails to update, a system at the DMV or a school or hospital goes down, and so on.