Do you think stronger versions of the result mentioned by mishka might be able to count as “real” continuous learning, if the ‘mutable state’ (e.g. the LoRA adapter) has enough capacity to exhibit the gains in capabilities?
E.g. in the ‘country of geniuses in the datacenter’ model, while a bunch of LLMs together would not be able to develop new fields, it seems at least possible for a bunch of LLMs augmented with large ‘mutable states’ (such as LoRA adapters) to do so, at least to the limit of new things ‘within’ the capacity of the ‘fixed baseweight + learned LoRA space’. Current LoRA adapters in use are too small for that, but one could think of a much larger ones (including ones with just as many parameters as current models have ‘weights’).
Now, suppose that I take a generic imitation-learning algorithm (e.g. self-supervised learning in a transformer-architecture neural net, just like LLM pretraining), and have it watch a deep Q network play Atari Breakout, as it starts from random initialization, and gets better and better over 1M iterations. (...)
Question: Is this trained imitation-learner actually a good imitation of the deep Q network?
I am not familiar with the training procedure mentioned by mishka to train hypernetworks. But if it does work, might it not make it possible to ‘imitate the learning updates from true learners’—maybe from watching 10,000 deep Q networks play thousands of games instead—and then apply such updates recursively?
Do you think stronger versions of the result mentioned by mishka might be able to count as “real” continuous learning, if the ‘mutable state’ (e.g. the LoRA adapter) has enough capacity to exhibit the gains in capabilities?
E.g. in the ‘country of geniuses in the datacenter’ model, while a bunch of LLMs together would not be able to develop new fields, it seems at least possible for a bunch of LLMs augmented with large ‘mutable states’ (such as LoRA adapters) to do so, at least to the limit of new things ‘within’ the capacity of the ‘fixed baseweight + learned LoRA space’. Current LoRA adapters in use are too small for that, but one could think of a much larger ones (including ones with just as many parameters as current models have ‘weights’).
I am not familiar with the training procedure mentioned by mishka to train hypernetworks. But if it does work, might it not make it possible to ‘imitate the learning updates from true learners’—maybe from watching 10,000 deep Q networks play thousands of games instead—and then apply such updates recursively?