There’s no doubt a world simulator of some sort is probably going to be an important component in any AGI, at the very least for planning—Yan LeCun has talked about this a lot. There’s also this work where they show a VAE type thing can be configured to run internal simulations of the environment it was trained on.
In brief, a few issues I see here:
You haven’t actually provided any evidence that GPT does simulation other than “Just saying “this AI is a simulator” naturalizes many of the counterintuitive properties of GPT which don’t usually become apparent to people until they’ve had a lot of hands-on experience with generating text.” What counterintuitve properties, exactly? Examples I’ve seen show GPT-3 is not simulating the environment being described in the text. I’ve seen a lot impressive examples too, but I find it hard to draw conclusions on how the model works by just reading lots and lots of outputs… I wonder what experiments could be done to test your idea that it’s running a simulation.
Even for very simple to simulate processes such as addition or symbol substitution, GPT has, in my view, trouble learning them, even though it does Grok those things eventually. For things like multiplication, the accuracy it has depends on how often the numbers appear in the training data (https://arxiv.org/abs/2202.07206), which is a bit telling, I think.
Simulating the laws of physics is really hard.. trust me on this (I did a Ph.D. in molecular dynamics simulation). If it’s doing any simulation at all, it’s got to be some high level heuristic type stuff. If it’s really good, it might be capable of simulating basic geometric constraints (although IIRC GPT is superb at spatial reasoning). Even humans are really bad at properly simulating physics accurately (researchers found that most people do really poorly on a test of basic physics based reasoning, like basic kinematics (will this ball curve left, right , or go straight, etc)). I imagine gradient descent is going to be much more likely to settle on shortcut rules and heuristics rather than implementing a complex simulation.
my impression is that by simulator and simulacra this post is not intending to claim that the thing it is simulating is realphysics but rather that it learns a general “textphysics engine”, the model, which runs textphysics environments. it’s essentially just a reframing of the prediction objective to describe deployment time—not a claim that the model actually learns a strong causal simplification of the full variety of real physics.
Even if it did learn microscopic physics, the knowledge wouldn’t be of use for most text predictions because the input doesn’t specify/determine microscopic state information. It is forced by the partially observed state to simulate at a higher level of abstraction than microphysics—it must treat the input as probabilistic evidence for unobserved variables that affect time evolution.
There’s no doubt a world simulator of some sort is probably going to be an important component in any AGI, at the very least for planning—Yan LeCun has talked about this a lot. There’s also this work where they show a VAE type thing can be configured to run internal simulations of the environment it was trained on.
In brief, a few issues I see here:
You haven’t actually provided any evidence that GPT does simulation other than “Just saying “this AI is a simulator” naturalizes many of the counterintuitive properties of GPT which don’t usually become apparent to people until they’ve had a lot of hands-on experience with generating text.” What counterintuitve properties, exactly? Examples I’ve seen show GPT-3 is not simulating the environment being described in the text. I’ve seen a lot impressive examples too, but I find it hard to draw conclusions on how the model works by just reading lots and lots of outputs… I wonder what experiments could be done to test your idea that it’s running a simulation.
Even for very simple to simulate processes such as addition or symbol substitution, GPT has, in my view, trouble learning them, even though it does Grok those things eventually. For things like multiplication, the accuracy it has depends on how often the numbers appear in the training data (https://arxiv.org/abs/2202.07206), which is a bit telling, I think.
Simulating the laws of physics is really hard.. trust me on this (I did a Ph.D. in molecular dynamics simulation). If it’s doing any simulation at all, it’s got to be some high level heuristic type stuff. If it’s really good, it might be capable of simulating basic geometric constraints (although IIRC GPT is superb at spatial reasoning). Even humans are really bad at properly simulating physics accurately (researchers found that most people do really poorly on a test of basic physics based reasoning, like basic kinematics (will this ball curve left, right , or go straight, etc)). I imagine gradient descent is going to be much more likely to settle on shortcut rules and heuristics rather than implementing a complex simulation.
my impression is that by simulator and simulacra this post is not intending to claim that the thing it is simulating is realphysics but rather that it learns a general “textphysics engine”, the model, which runs textphysics environments. it’s essentially just a reframing of the prediction objective to describe deployment time—not a claim that the model actually learns a strong causal simplification of the full variety of real physics.
That’s correct.
Even if it did learn microscopic physics, the knowledge wouldn’t be of use for most text predictions because the input doesn’t specify/determine microscopic state information. It is forced by the partially observed state to simulate at a higher level of abstraction than microphysics—it must treat the input as probabilistic evidence for unobserved variables that affect time evolution.
See this comment for slightly more elaboration.