It seems to me like a big problem with this approach is that it’s horribly compute inefficient to train agents entirely within a simulation, compared to training models on human data. (Apologies if you addressed this in the post and I missed it)
I don’t see how training of VPT or EfficientZero was compute inefficient. In fact for self driving cars the exact opposite is true—training in simulation can be much more efficient then training in reality.
VPT and EfficientZero are trained in toy environments, and self driving cars sims are also low-dimensional hard-coded approximations of the deployment domain (which afaik does cause some problems for edge cases in the real world).
The sim for training AGI will probably have to be a rich domain, which is more computationally intensive to simulate and so will probably require lazy rendering like you say in the post, but lazy rendering runs into challenges of world consistency.
Right now we can lazily simulate rich domains with GPT but they’re difficult to program reliably and not autonomously stable (though I think they’ll become much more autonomously stable soon). And the richness of current GPT simulations inherits from massive human datasets. Human datasets are convenient because you have some guaranteed samples of a rich and coherent world. GPTs bootstrap from the optimization done by evolution and thousands of years of culture compressing world knowledge and cognitive algorithms into an efficient code, language. Skipping this step it’s a lot less clear how you’d train AGI, and it seems to me barring some breakthrough on the nature of intelligence or efficient ML it would have to be more computationally intensive to compensate for the disadvantage of starting tabula rasa.
Multiplayer minecraft may already be a complex enough environment for AGI, even if it is a ‘toy’ world in terms of visuals. Regardless even if AGI requires an environment with more realistic and complex physics such simulations are not expensive relative to AGI itself. “Lazy rendering” of the kind we’d want to use for more advanced sims does not have any inherent consistency tradeoff beyond those inherent to any practical approximate simulation physics.
Foundation text and vision models will soon begin to transform sim/games but that is mostly a separate issue from agent use of language.
Naturally AGI will require language; any sim-grown agents would be taught language, but that doesn’t imply they need to learn language via absorbing the internet like GPT.
I agree minecraft is a complex enough environment for AGI in principle. Perhaps rich domain distinction wasn’t the right distinction. It’s more like whether there are already abstractions adapted to intelligence built into the environment or not, like human language. Game of Life is expressive enough to be an environment for AGI in principle too, but it’s not clear how to go about that.
Naturally AGI will require language; any sim-grown agents would be taught language, but that doesn’t imply they need to learn language via absorbing the internet like GPT.
That’s certainly true, but it seems like currently an unsolved problem how to make sim-grown agents that learn a language from scratch. That’s my point: brute force search such as evolutionary algorithms would require much more compute.
In my view—and not everyone agrees with this, but many do—GPT is the only instance of (proto-) artificial general intelligence we’ve created. This makes sense because it bootstraps off human intelligence, including the cultural/memetic layer, which was forged by eons of optimization in rich multi agent environments. Self-supervised learning on human data is the low hanging fruit. Even more so if the target is not just “smart general optimizer” but something that resembles human intelligence in all the other ways, such as using something recognizable as language and more generally being comprehensible to us at all.
I don’t think of LLMs like GPT3 as agents that uses language; they are artificial linguistic cortices which can be useful to brains as (external or internal) tools.
I imagine that a more ‘true’ AGI system will be somewhat brain-like in that it will develop a linguistic cortex purely through embedded active learning in a social environment, but is much more than just that one module—even if that particular module is the key enabler for human-like intelligence as distinct from animal intelligence.
That’s certainly true, but it seems like currently an unsolved problem how to make sim-grown agents that learn a language from scratch.
I find this statement puzzling because it is rather obvious how to build sim-grown agents that learn language from scratch. You simply need to replicate something like a human child’s development environment and train a sufficiently powerful/general model there. This probably requires a sim where the child is immersed in adults conversing with it and themselves, a sufficiently complex action space, etc. That probably hasn’t been done yet partly because nobody has bothered to try (10 years of data at least, perhaps via a thousand volunteers contributing a couple weeks?) and also perhaps because current systems don’t have the capacity/capability to learn that quickly enough for various reasons.
The game/sim path to AGI—which is more or less deepmind’s traditional approach—probably goes through animal-like intelligence first, and arguably things like VPT are already getting close. That of course is not the only path: there’s also a prosaic GPT3 style path where you build out individual specialized modules first and then gradually integrate them.
It seems to me like a big problem with this approach is that it’s horribly compute inefficient to train agents entirely within a simulation, compared to training models on human data. (Apologies if you addressed this in the post and I missed it)
I don’t see how training of VPT or EfficientZero was compute inefficient. In fact for self driving cars the exact opposite is true—training in simulation can be much more efficient then training in reality.
VPT and EfficientZero are trained in toy environments, and self driving cars sims are also low-dimensional hard-coded approximations of the deployment domain (which afaik does cause some problems for edge cases in the real world).
The sim for training AGI will probably have to be a rich domain, which is more computationally intensive to simulate and so will probably require lazy rendering like you say in the post, but lazy rendering runs into challenges of world consistency.
Right now we can lazily simulate rich domains with GPT but they’re difficult to program reliably and not autonomously stable (though I think they’ll become much more autonomously stable soon). And the richness of current GPT simulations inherits from massive human datasets. Human datasets are convenient because you have some guaranteed samples of a rich and coherent world. GPTs bootstrap from the optimization done by evolution and thousands of years of culture compressing world knowledge and cognitive algorithms into an efficient code, language. Skipping this step it’s a lot less clear how you’d train AGI, and it seems to me barring some breakthrough on the nature of intelligence or efficient ML it would have to be more computationally intensive to compensate for the disadvantage of starting tabula rasa.
Multiplayer minecraft may already be a complex enough environment for AGI, even if it is a ‘toy’ world in terms of visuals. Regardless even if AGI requires an environment with more realistic and complex physics such simulations are not expensive relative to AGI itself. “Lazy rendering” of the kind we’d want to use for more advanced sims does not have any inherent consistency tradeoff beyond those inherent to any practical approximate simulation physics.
Foundation text and vision models will soon begin to transform sim/games but that is mostly a separate issue from agent use of language.
Naturally AGI will require language; any sim-grown agents would be taught language, but that doesn’t imply they need to learn language via absorbing the internet like GPT.
I agree minecraft is a complex enough environment for AGI in principle. Perhaps rich domain distinction wasn’t the right distinction. It’s more like whether there are already abstractions adapted to intelligence built into the environment or not, like human language. Game of Life is expressive enough to be an environment for AGI in principle too, but it’s not clear how to go about that.
That’s certainly true, but it seems like currently an unsolved problem how to make sim-grown agents that learn a language from scratch. That’s my point: brute force search such as evolutionary algorithms would require much more compute.
In my view—and not everyone agrees with this, but many do—GPT is the only instance of (proto-) artificial general intelligence we’ve created. This makes sense because it bootstraps off human intelligence, including the cultural/memetic layer, which was forged by eons of optimization in rich multi agent environments. Self-supervised learning on human data is the low hanging fruit. Even more so if the target is not just “smart general optimizer” but something that resembles human intelligence in all the other ways, such as using something recognizable as language and more generally being comprehensible to us at all.
I don’t think of LLMs like GPT3 as agents that uses language; they are artificial linguistic cortices which can be useful to brains as (external or internal) tools.
I imagine that a more ‘true’ AGI system will be somewhat brain-like in that it will develop a linguistic cortex purely through embedded active learning in a social environment, but is much more than just that one module—even if that particular module is the key enabler for human-like intelligence as distinct from animal intelligence.
I find this statement puzzling because it is rather obvious how to build sim-grown agents that learn language from scratch. You simply need to replicate something like a human child’s development environment and train a sufficiently powerful/general model there. This probably requires a sim where the child is immersed in adults conversing with it and themselves, a sufficiently complex action space, etc. That probably hasn’t been done yet partly because nobody has bothered to try (10 years of data at least, perhaps via a thousand volunteers contributing a couple weeks?) and also perhaps because current systems don’t have the capacity/capability to learn that quickly enough for various reasons.
The game/sim path to AGI—which is more or less deepmind’s traditional approach—probably goes through animal-like intelligence first, and arguably things like VPT are already getting close. That of course is not the only path: there’s also a prosaic GPT3 style path where you build out individual specialized modules first and then gradually integrate them.