One related thing I was thinking about last week: part of the idea of abstraction is that we can pick a Markov blanket around some variable X, and anything outside that Markov blanket can only “see” abstract summary information f(X). So, if we have a goal which only cares about things outside that Markov blanket, then that goal will only care about f(X) rather than all of X. This holds for any goal which only cares about things outside the blanket. That sounds like instrumental convergence: any goal which does not explicitly care about things near X itself, will care only about controlling f(X), not all of X.
This isn’t quite the same notion of goal-locality that the OP is using (it’s not about how close the goal-variables are to the agent), but it feels like there’s some overlapping ideas there.
The more I think about it, the more I come to believe that locality is very related to abstraction. Not the distance part necessarily, but the underlying intuition. If my goal is not “about the world”, then I can throw almost all information about the world except a few details and still be able to check my goal. The “world” of the thermostat is in that sense a very abstracted map of the world where anything except the number on its sensor is thrown away.
Thanks! Glad that I managed to write something that was not causally or rhetorically all wrong. ^^
One related thing I was thinking about last week: part of the idea of abstraction is that we can pick a Markov blanket around some variable X, and anything outside that Markov blanket can only “see” abstract summary information f(X). So, if we have a goal which only cares about things outside that Markov blanket, then that goal will only care about f(X) rather than all of X
That makes even more sense to me than you might think. My intuitions about locality comes from its uses in distributed computing, where it measures both how many rounds of communication are needed to solve a problem and how far in the communication graph one needs to look to compute one’s own output. This looks like my use of locality here.
On the other hand, recent work on distributed complexity also studied the volume complexity of a problem: the size of the subgraph one needs to look at, which might be very different from a ball. The only real constraint is connectedness. Modulo the usual “exactness issue”, which we can deal with by replacing “the node is not used” by “only f(X) is used”, this looks a lot like your idea.
Nice post!
One related thing I was thinking about last week: part of the idea of abstraction is that we can pick a Markov blanket around some variable X, and anything outside that Markov blanket can only “see” abstract summary information f(X). So, if we have a goal which only cares about things outside that Markov blanket, then that goal will only care about f(X) rather than all of X. This holds for any goal which only cares about things outside the blanket. That sounds like instrumental convergence: any goal which does not explicitly care about things near X itself, will care only about controlling f(X), not all of X.
This isn’t quite the same notion of goal-locality that the OP is using (it’s not about how close the goal-variables are to the agent), but it feels like there’s some overlapping ideas there.
The more I think about it, the more I come to believe that locality is very related to abstraction. Not the distance part necessarily, but the underlying intuition. If my goal is not “about the world”, then I can throw almost all information about the world except a few details and still be able to check my goal. The “world” of the thermostat is in that sense a very abstracted map of the world where anything except the number on its sensor is thrown away.
Thanks! Glad that I managed to write something that was not causally or rhetorically all wrong. ^^
That makes even more sense to me than you might think. My intuitions about locality comes from its uses in distributed computing, where it measures both how many rounds of communication are needed to solve a problem and how far in the communication graph one needs to look to compute one’s own output. This looks like my use of locality here.
On the other hand, recent work on distributed complexity also studied the volume complexity of a problem: the size of the subgraph one needs to look at, which might be very different from a ball. The only real constraint is connectedness. Modulo the usual “exactness issue”, which we can deal with by replacing “the node is not used” by “only f(X) is used”, this looks a lot like your idea.