Another reason for thinking that LLM AGI will have memory/state, conditional on AGI being built, is that it’s probably the only blocker to something like drop-in remote workers being built, and from there escalating to AGI and ASI because it would allow for potentially unbounded meta-learning given unbounded resources, and even make meta-learning in general far more effective for longer time periods.
Gwern explains why meta-learning explains basically all of the baffling LLM weaknesses here, and the short version is that right now, LLM weights are frozen after training and they have zero neuroplasticity after training (modulo in-context learning, but that is way too weak to matter), and this means LLMs can learn 0 new tricks after release, and in all but the simplest tasks, it turns out that learning has to be continuously there, which was the key thing we didn’t really realize was a limitation of GPT-N style AIs.
Good timing—the day after you posted this, a round of new Tom & Jerry cartoons swept through twitter, fueled by transformer models which included in their layers MLPs that can learn at test time. Github repo here: https://github.com/test-time-training (The videos are more eye-catching, but they’ve also done text models).
Also notable: the big OpenAI reveal today was some sort of better personalization. Instead of the crude ‘saved facts’ personalization ChatGPT has had for a long time and which has never made much of a difference, they’re doing… something. Unclear if it’s merely RAG or if they are also doing something interesting like lightweight finetuning. But the GPTs definitely seem to have much better access to your other sessions in the web interface, and as far as I know, few other interfaces with frontier models have tried to do much personalization, so this will be an interesting real-world test at scale about how much simple personalization can help with LLMs (similar to Midjourney’s relatively new personalization feature, which I get a lot out of).
It’s almost certainly untrue that OpenAI is doing per-user finetuning for personalization.
Each user-tuned model (or even a LoRA adapter) must stick to the node where it was trained. You lose the ability to route traffic to any idle instance. OpenAI explicitly optimises for pooling capacity across huge fleets; tying sessions to devices would crater utilisation.
I agree at this point: it is not per-user finetuning. The personalization has been prodded heavily, and it seems to boil down to a standard RAG interface plus a slightly interesting ‘summarization’ approach to try to describe the user in text (as opposed to a ‘user embedding’ or something else). I have not seen any signs of either lightweight or full finetuning, and several observations strongly cut against it: for example, users describe a ‘discrete’ behavior where the current GPT either knows something from another session, or it doesn’t, but it is never ‘in between’, and it only seems to draw on a few other sessions at any time; this points to RAG as the workhorse (the relevant other snippet either got retrieved or it didn’t), rather than any kind of finetuning where you would expect ‘fuzzy’ recall and signs of information leaking in from all recent sessions.
Perhaps for that reason, it has not made a big impact (at least once people got over the narcissistic rush of asking GPT about the summary of you, either flatteringly sycophantic or not). It presumably is quietly helping behind the scenes, but I haven’t noticed any clear big benefits to it. (And there are some drawbacks.)
Yeah, this sort of thing, if it actually scales and can be adapted to other paradigms (like putting an RNN or transformers), would be the final breakthrough sufficient for AGI, because as I’ve said, one of the things that keeps LLM agents from being better is their inability to hold memory/state, which cripples meta-learning (without expensive compute investment), and this new paper is possibly a first step towards the return of recurrence/RNN architectures.
That’s very useful, thanks! That’s exactly the argument I was trying to make here. I didn’t use the term drop-in remote worker but that’s the economic incentive I’m addressing (among more immediate ones- I think large incentives start long before you have a system that can learn any job).
Lack of episodic memory looks to me like the primary reason LLMs have weaknesses humans do not. The other is a well-developed skillset for managing complex trains of thought. o1 and o3 and maybe the other reasoning models have learned some of that skillset but only mastered it in the narrow domains that allowed training on verifiable answers. Scaffolding and/or training for executive function (thought management) and/or memory seems poised to increase the growth rate of long time-horizon task performance. It’s going to take some work still but I don’t think it’s wise to assume that the seven-month doubling period won’t speed up, or that some point it will just jump to infinity, while the complexity of the necessary subtasks is still a limiting factor.
Humans don’t train on tons of increasingly long tasks, we just learn some strategies and some skills for managing our thought, like checking carefully whether a step has been accomplished, searching memory for task structure and where we’re at in the plan if we lose our place, etc. Humans are worse at longer tasks, but any normal adult human can tackle a task of any length and at least keep getting better at it for as long as they decide to stick with it.
Another reason for thinking that LLM AGI will have memory/state, conditional on AGI being built, is that it’s probably the only blocker to something like drop-in remote workers being built, and from there escalating to AGI and ASI because it would allow for potentially unbounded meta-learning given unbounded resources, and even make meta-learning in general far more effective for longer time periods.
Gwern explains why meta-learning explains basically all of the baffling LLM weaknesses here, and the short version is that right now, LLM weights are frozen after training and they have zero neuroplasticity after training (modulo in-context learning, but that is way too weak to matter), and this means LLMs can learn 0 new tricks after release, and in all but the simplest tasks, it turns out that learning has to be continuously there, which was the key thing we didn’t really realize was a limitation of GPT-N style AIs.
More in the comment below:
https://www.lesswrong.com/posts/deesrjitvXM4xYGZd/metr-measuring-ai-ability-to-complete-long-tasks#hSkQG2N8rkKXosLEF
Good timing—the day after you posted this, a round of new Tom & Jerry cartoons swept through twitter, fueled by transformer models which included in their layers MLPs that can learn at test time. Github repo here: https://github.com/test-time-training (The videos are more eye-catching, but they’ve also done text models).
Also notable: the big OpenAI reveal today was some sort of better personalization. Instead of the crude ‘saved facts’ personalization ChatGPT has had for a long time and which has never made much of a difference, they’re doing… something. Unclear if it’s merely RAG or if they are also doing something interesting like lightweight finetuning. But the GPTs definitely seem to have much better access to your other sessions in the web interface, and as far as I know, few other interfaces with frontier models have tried to do much personalization, so this will be an interesting real-world test at scale about how much simple personalization can help with LLMs (similar to Midjourney’s relatively new personalization feature, which I get a lot out of).
It’s almost certainly untrue that OpenAI is doing per-user finetuning for personalization.
Each user-tuned model (or even a LoRA adapter) must stick to the node where it was trained. You lose the ability to route traffic to any idle instance. OpenAI explicitly optimises for pooling capacity across huge fleets; tying sessions to devices would crater utilisation.
I agree at this point: it is not per-user finetuning. The personalization has been prodded heavily, and it seems to boil down to a standard RAG interface plus a slightly interesting ‘summarization’ approach to try to describe the user in text (as opposed to a ‘user embedding’ or something else). I have not seen any signs of either lightweight or full finetuning, and several observations strongly cut against it: for example, users describe a ‘discrete’ behavior where the current GPT either knows something from another session, or it doesn’t, but it is never ‘in between’, and it only seems to draw on a few other sessions at any time; this points to RAG as the workhorse (the relevant other snippet either got retrieved or it didn’t), rather than any kind of finetuning where you would expect ‘fuzzy’ recall and signs of information leaking in from all recent sessions.
Perhaps for that reason, it has not made a big impact (at least once people got over the narcissistic rush of asking GPT about the summary of you, either flatteringly sycophantic or not). It presumably is quietly helping behind the scenes, but I haven’t noticed any clear big benefits to it. (And there are some drawbacks.)
Yeah, this sort of thing, if it actually scales and can be adapted to other paradigms (like putting an RNN or transformers), would be the final breakthrough sufficient for AGI, because as I’ve said, one of the things that keeps LLM agents from being better is their inability to hold memory/state, which cripples meta-learning (without expensive compute investment), and this new paper is possibly a first step towards the return of recurrence/RNN architectures.
That’s very useful, thanks! That’s exactly the argument I was trying to make here. I didn’t use the term drop-in remote worker but that’s the economic incentive I’m addressing (among more immediate ones- I think large incentives start long before you have a system that can learn any job).
Lack of episodic memory looks to me like the primary reason LLMs have weaknesses humans do not. The other is a well-developed skillset for managing complex trains of thought. o1 and o3 and maybe the other reasoning models have learned some of that skillset but only mastered it in the narrow domains that allowed training on verifiable answers. Scaffolding and/or training for executive function (thought management) and/or memory seems poised to increase the growth rate of long time-horizon task performance. It’s going to take some work still but I don’t think it’s wise to assume that the seven-month doubling period won’t speed up, or that some point it will just jump to infinity, while the complexity of the necessary subtasks is still a limiting factor.
Humans don’t train on tons of increasingly long tasks, we just learn some strategies and some skills for managing our thought, like checking carefully whether a step has been accomplished, searching memory for task structure and where we’re at in the plan if we lose our place, etc. Humans are worse at longer tasks, but any normal adult human can tackle a task of any length and at least keep getting better at it for as long as they decide to stick with it.