My own take is that I’m fairly sympathetic to the “LLMs are already able to get to AGI” view, with the caveat that most of the difference between human and LLM learning where humans are superior than LLMs comes from being able to do meta-learning over long horizons, and we haven’t yet been shown this is possible for LLMs to do purely by scaling compute.
Indeed, I think it’s the entire crux of the scaling hypothesis debate, in whether scale enables meta-learning over longer and longer time periods:
My own take is that I’m fairly sympathetic to the “LLMs are already able to get to AGI” view, with the caveat that most of the difference between human and LLM learning where humans are superior than LLMs comes from being able to do meta-learning over long horizons, and we haven’t yet been shown this is possible for LLMs to do purely by scaling compute.
Indeed, I think it’s the entire crux of the scaling hypothesis debate, in whether scale enables meta-learning over longer and longer time periods:
https://www.lesswrong.com/posts/deesrjitvXM4xYGZd/metr-measuring-ai-ability-to-complete-long-tasks#hSkQG2N8rkKXosLEF