There’s another whole route to scaling LLM capabilities. LLMs themselves might not need to scale up much more at all to enable AGI. Scaffolding LLMs to them to make language model cognitive architectures can fill at least some of the gaps in their abilities. Many groups are working on this. The Minecraft agent JARVIS-1 is the biggest success so far, which is pretty limited; but it’s hard to guess how far this approach will get. Opinions are split. I describe some reasons they could make nonlinear progress with better memory systems (and other enhancements) in Capabilities and alignment of LLM cognitive architectures.
This possiblity is alarming, but it’s not necessarily a bad thing. LMCAs seem to be inherently easier to align than any other AGI design I know of with a real chance of being first-past-the-post. Curiously, I haven’t been able to drum up much interest in this, despite it being potentially a short timeline, so urgent, and also quite possibly our best shot at aligned AGI. I haven’t even gotten criticisms shooting it down, just valid arguments that it’s not guaranteed to work.
I’ve also noticed that scaffolded LLM agents seem inherently safer. In particular, deceptive alignment would be hard for one such agent to achieve, if at every thought-step it has to reformulate its complete mind state into the English language just in order to think at all.
You might be interested in some work done by the ARC Evals team, who prioritize this type of agent for capability testing.
There’s another whole route to scaling LLM capabilities. LLMs themselves might not need to scale up much more at all to enable AGI. Scaffolding LLMs to them to make language model cognitive architectures can fill at least some of the gaps in their abilities. Many groups are working on this. The Minecraft agent JARVIS-1 is the biggest success so far, which is pretty limited; but it’s hard to guess how far this approach will get. Opinions are split. I describe some reasons they could make nonlinear progress with better memory systems (and other enhancements) in Capabilities and alignment of LLM cognitive architectures.
This possiblity is alarming, but it’s not necessarily a bad thing. LMCAs seem to be inherently easier to align than any other AGI design I know of with a real chance of being first-past-the-post. Curiously, I haven’t been able to drum up much interest in this, despite it being potentially a short timeline, so urgent, and also quite possibly our best shot at aligned AGI. I haven’t even gotten criticisms shooting it down, just valid arguments that it’s not guaranteed to work.
I’ve also noticed that scaffolded LLM agents seem inherently safer. In particular, deceptive alignment would be hard for one such agent to achieve, if at every thought-step it has to reformulate its complete mind state into the English language just in order to think at all.
You might be interested in some work done by the ARC Evals team, who prioritize this type of agent for capability testing.