If you’re the sort of thing that skillfully generates and enacts long-term plans, and you’re the sort of planner that sticks to its guns and finds a way to succeed in the face of the many obstacles the real world throws your way (rather than giving up or wandering off to chase some new shiny thing every time a new shiny thing comes along), then the way I think about these things, it’s a little hard to imagine that you don’t contain some reasonably strong optimization that strategically steers the world into particular states.
It seems this post has maybe mixed “generating” with “enacting”. Currently, it seems LLMs only attempt the former during prediction. In general terms, predicting a long-horizon-actor’s reasoning is implicit in the task of myopically predicting the next thing that actor would do. For a specific example, you could imagine a model predicting the next move of a grandmaster’s or stockfish’s chess game (or text in an author’s book, or an industrial project description, to use your longer-horizon examples).
The first paragraph of /u/paulfchristiano’s response might be getting at something similar, but it seems worth saying this directly.[1]
(This also seems like a basic point, so I wonder if I misunderstood the post.. but it seems like something isomorphic to it is in the top comment, so I’m not sure.)
It seems this post has maybe mixed “generating” with “enacting”. Currently, it seems LLMs only attempt the former during prediction. In general terms, predicting a long-horizon-actor’s reasoning is implicit in the task of myopically predicting the next thing that actor would do. For a specific example, you could imagine a model predicting the next move of a grandmaster’s or stockfish’s chess game (or text in an author’s book, or an industrial project description, to use your longer-horizon examples).
The first paragraph of /u/paulfchristiano’s response might be getting at something similar, but it seems worth saying this directly.[1]
(This also seems like a basic point, so I wonder if I misunderstood the post.. but it seems like something isomorphic to it is in the top comment, so I’m not sure.)