your point-of-view … LLM-based AI … lacks continuous learning and sample efficiency
Sample efficiency is not a problem with RLVR, LLMs can reach or exceed world class human capability (at any given sufficiently specific thing) with a reasonable amount of training data in the form of tasks/environments/graders. And this will probably get automated (within the remaining runway of raw scaling for LLMs), with slow prosaic RSI of LLMs automatically preparing such training data and using it to train the next model. Continual learning in the sense of unlimited context is not helpful because in-context learning doesn’t train deep skills (LLMs can’t learn to play chess well by looking at memory notes in their context).
The crucial constraint is that retraining with RLVR in the current paradigm (by preparing the next model) happens too slowly, much slower than token generation. Amdahl’s law then destroys the advantage of 100x faster idea generation and problem solving (using the current skills and understanding) in the overall task of research, because the subtask of learning novel deep skills doesn’t have a speedup compared to humans (and in fact might happen much slower). Raw intelligence doesn’t get around the lack of a speedup, because LLMs won’t get to vastly superhuman levels within the remaining runway of scaling in the next few years.
So continual learning in the sense of doing something like RLVR on the fly would indeed solve this hobbling, but it’s currently an unsolved problem. If LLMs can’t do conceptual research faster than humans because of their long learning loop (even if the learning becomes automated in a general way, making LLMs able to eventually learn any specific skill), and they aren’t vastly superhuman in coming up with ideas far outside their current skills and understanding, then they won’t be crucial in solving this hobbling either.
Sample efficiency is not a problem with RLVR, LLMs can reach or exceed world class human capability (at any given sufficiently specific thing) with a reasonable amount of training data in the form of tasks/environments/graders. And this will probably get automated (within the remaining runway of raw scaling for LLMs), with slow prosaic RSI of LLMs automatically preparing such training data and using it to train the next model. Continual learning in the sense of unlimited context is not helpful because in-context learning doesn’t train deep skills (LLMs can’t learn to play chess well by looking at memory notes in their context).
The crucial constraint is that retraining with RLVR in the current paradigm (by preparing the next model) happens too slowly, much slower than token generation. Amdahl’s law then destroys the advantage of 100x faster idea generation and problem solving (using the current skills and understanding) in the overall task of research, because the subtask of learning novel deep skills doesn’t have a speedup compared to humans (and in fact might happen much slower). Raw intelligence doesn’t get around the lack of a speedup, because LLMs won’t get to vastly superhuman levels within the remaining runway of scaling in the next few years.
So continual learning in the sense of doing something like RLVR on the fly would indeed solve this hobbling, but it’s currently an unsolved problem. If LLMs can’t do conceptual research faster than humans because of their long learning loop (even if the learning becomes automated in a general way, making LLMs able to eventually learn any specific skill), and they aren’t vastly superhuman in coming up with ideas far outside their current skills and understanding, then they won’t be crucial in solving this hobbling either.