I made the massive mistake of reading this yesterday and waiting until today to write my summary, and that was pretty bad for my retention
Nevertheless, here it is. I think I’m most curious about whether people here think that my last three bullet points in the opinion/confusions are roughly correct.
Summary In this article, John uses his causal model approach to agent foundations to explain what “actions” are, with the goal to solve some conceptual problems of optimization along the way. Concretely, imagine you have a causal model and separated it into “agent” and “environment”.First, he declares that to “optimize” something means to steer variable values “far away” (in space or time) to particular values. By the so-called telephone theorem, far-apart things in such a model only interact via an “arbitrarily well-conserved abstract summary” of variable values. He defines the abstract summary of what’s going out of the agent as “the actions” and the abstract summary of what comes from the environment as “the observations”. This means that actions and observations are conceptually “the things that optimize something”. This has a few neat philosophical advantages over defining everything going out of an agent as its actions and as “optimizing something”:
Very low-level interactions like hair-molecule movements will not contribute to the abstract summary because such information is not preserved. Thus they will not be part of the “actions”. This is intuitive.
Under this definition, not *every* entity in the causal model will be optimizing something. We don’t want to say that a rock optimizes something, and indeed, it doesn’t: its interactions with the environment are completely thermodynamic and fizzle out over a distance.
This gives also some flexibility about the choice of a cartesian boundary of an agent. Want to add a car to a human? You can do this now since the actions = summaries are the same when looking at a sufficient distance.
Note that optimizing yourself is still optimization, even though you yourself seem quite close intuitively: this is since you’re actually optimizing yourself in the future, and your future self is far away.
Opinion/Confusions At some point, I should probably understand the telephone theorem better. Additionally, I’m not yet exactly sure if I think in the right way about the concept relationships in this post. Is the following roughly correct?
Agents can perform actions, but non-agents cannot perform actions. It is thus not allowed to label everything as an agent.
Whether something is an action is not a yes/no question. The collection of values of a variable cluster is “more action-like” if it provides “more preserved information further away”
Actions generally optimize something since “containing information far away” precisely means that it “influences variable values far away”. (Here I’m not sure: If I send out a light signal with a laser into the universe, and this laser contains the information of the bible, then this seems a quite content-rich abstract summary. This laser beam may not hit anything. Does it still count as “optimizing something” since it perfectly propagates its own information content into its own future state? I think yes, but it’s worth asking)
I think the article thus tries to bundle “agents”, “actions”, “information at a distance” and “optimization” into one concept by saying:
“agents are the things that can perform actions”;
“actions are the things with abstract summaries at a distance containing lots of information”;
“large information at a distance equals optimization”.
I made the massive mistake of reading this yesterday and waiting until today to write my summary, and that was pretty bad for my retention
I think I’m most curious about whether people here think that my last three bullet points in the opinion/confusions are roughly correct.
Summary
In this article, John uses his causal model approach to agent foundations to explain what “actions” are, with the goal to solve some conceptual problems of optimization along the way.
Concretely, imagine you have a causal model and separated it into “agent” and “environment”.First, he declares that to “optimize” something means to steer variable values “far away” (in space or time) to particular values. By the so-called telephone theorem, far-apart things in such a model only interact via an “arbitrarily well-conserved abstract summary” of variable values. He defines the abstract summary of what’s going out of the agent as “the actions” and the abstract summary of what comes from the environment as “the observations”. This means that actions and observations are conceptually “the things that optimize something”.
This has a few neat philosophical advantages over defining everything going out of an agent as its actions and as “optimizing something”:
Very low-level interactions like hair-molecule movements will not contribute to the abstract summary because such information is not preserved. Thus they will not be part of the “actions”. This is intuitive.
Under this definition, not *every* entity in the causal model will be optimizing something. We don’t want to say that a rock optimizes something, and indeed, it doesn’t: its interactions with the environment are completely thermodynamic and fizzle out over a distance.
This gives also some flexibility about the choice of a cartesian boundary of an agent. Want to add a car to a human? You can do this now since the actions = summaries are the same when looking at a sufficient distance.
Note that optimizing yourself is still optimization, even though you yourself seem quite close intuitively: this is since you’re actually optimizing yourself in the future, and your future self is far away.
Opinion/Confusions
At some point, I should probably understand the telephone theorem better. Additionally, I’m not yet exactly sure if I think in the right way about the concept relationships in this post. Is the following roughly correct?
Agents can perform actions, but non-agents cannot perform actions. It is thus not allowed to label everything as an agent.
Whether something is an action is not a yes/no question. The collection of values of a variable cluster is “more action-like” if it provides “more preserved information further away”
Actions generally optimize something since “containing information far away” precisely means that it “influences variable values far away”. (Here I’m not sure: If I send out a light signal with a laser into the universe, and this laser contains the information of the bible, then this seems a quite content-rich abstract summary. This laser beam may not hit anything. Does it still count as “optimizing something” since it perfectly propagates its own information content into its own future state? I think yes, but it’s worth asking)
I think the article thus tries to bundle “agents”, “actions”, “information at a distance” and “optimization” into one concept by saying:
“agents are the things that can perform actions”;
“actions are the things with abstract summaries at a distance containing lots of information”;
“large information at a distance equals optimization”.