I’m very interested to see what you discover or what framework for deconfusion you arrive at with this research project.
My own take on goal-directedness, to a first approximation, is that it is the property of a system that allows it to consistently arrive at a narrow region of state space from a broad range of starting points. The tighter the distribution of steady states, and the wider the distribution of starting states that allow the system to reach them, the more goal-directed it is. A system where a ball is always rolling down to the center of a basin could be considered more goal-directed than a ball rolling around a random landscape, for instance.
These goal states could exist within the agent itself (i.e., homeostatic set points) or out in the external environment (e.g., states that maximize attainable utility, like collecting resources). They could also be represented either explicitly as patterns within the agent’s mental model or implicitly within the structure of the agent’s policy functions.
Another dimension to this could be the ability to avoid unexpected states that would prevent the achievement of goals (e.g., avoid predators or move around obstacles), or the ability to select actions, either choosing among multiple narrow policies in pursuit of a single goal or choosing among multiple goals in pursuit of utility (or a meta-goal).
I like the idea about the size of the target states; there’s bound to be some interesting measure theory that I can apply if I decide to formalize in that direction. In fact, measure theory might be able to clarify some of the subtleties I alluded to above regarding what happens when we refine the world model (for example, in a way that causes a single goal state to split into two or more).
There are hints in your last paragraph of associating competence with goal-directedness, which I think is an association to avoid. For example, when a zebra is swimming across a river as fast as it can, I would like the extent to which that behaviour is considered goal-directed to be independent of whether that zebra is the one that gets attacked by a crocodile.
Maybe I could try to disentangle competence from goal-directedness in what I wrote. The main idea that I was trying to push in that paragraph is that there is more to goal-directed behavior in real animals than just movement toward a goal state. There is also (attempted) movement away from anti-goal states and around obstacle states.
An example of the former could be a zebra seeing a bunch of crocodiles congregated by the bank of the Nile and deciding not to cross the river today (unfortunately, it later got chased down and eaten by a lion due to the zebra’s incompetence at evading all anti-goal states).
An example of the latter could be a golfer veering his swing slightly to the right to avoid the sand traps on the left (unfortunately, the ball ended up landing in the pond instead due to the golfer’s incompetence at avoiding all obstacle states).
Anti-goals and obstacles act as repulsor states, complementing the attractor states known as goals, redirecting the flow of behavior to maximize the chances of survival and of reaching the actual goals.
As to the latter part of that paragraph, I think policy-selection for single goals and goal-selection more generally are important for enabling systems to exhibit flexible behavior. Someone in a recent thread (https://www.lesswrong.com/posts/3L46WGauGpr7nYubu/the-plan?commentId=nLCpJnxfaPzKXsbE2) brought up some interesting research on goal selection (more like goal pruning) in animals that could be worth looking into.
I’m very interested to see what you discover or what framework for deconfusion you arrive at with this research project.
My own take on goal-directedness, to a first approximation, is that it is the property of a system that allows it to consistently arrive at a narrow region of state space from a broad range of starting points. The tighter the distribution of steady states, and the wider the distribution of starting states that allow the system to reach them, the more goal-directed it is. A system where a ball is always rolling down to the center of a basin could be considered more goal-directed than a ball rolling around a random landscape, for instance.
These goal states could exist within the agent itself (i.e., homeostatic set points) or out in the external environment (e.g., states that maximize attainable utility, like collecting resources). They could also be represented either explicitly as patterns within the agent’s mental model or implicitly within the structure of the agent’s policy functions.
Another dimension to this could be the ability to avoid unexpected states that would prevent the achievement of goals (e.g., avoid predators or move around obstacles), or the ability to select actions, either choosing among multiple narrow policies in pursuit of a single goal or choosing among multiple goals in pursuit of utility (or a meta-goal).
Thanks for the ideas!
I like the idea about the size of the target states; there’s bound to be some interesting measure theory that I can apply if I decide to formalize in that direction. In fact, measure theory might be able to clarify some of the subtleties I alluded to above regarding what happens when we refine the world model (for example, in a way that causes a single goal state to split into two or more).
There are hints in your last paragraph of associating competence with goal-directedness, which I think is an association to avoid. For example, when a zebra is swimming across a river as fast as it can, I would like the extent to which that behaviour is considered goal-directed to be independent of whether that zebra is the one that gets attacked by a crocodile.
Maybe I could try to disentangle competence from goal-directedness in what I wrote. The main idea that I was trying to push in that paragraph is that there is more to goal-directed behavior in real animals than just movement toward a goal state. There is also (attempted) movement away from anti-goal states and around obstacle states.
An example of the former could be a zebra seeing a bunch of crocodiles congregated by the bank of the Nile and deciding not to cross the river today (unfortunately, it later got chased down and eaten by a lion due to the zebra’s incompetence at evading all anti-goal states).
An example of the latter could be a golfer veering his swing slightly to the right to avoid the sand traps on the left (unfortunately, the ball ended up landing in the pond instead due to the golfer’s incompetence at avoiding all obstacle states).
Anti-goals and obstacles act as repulsor states, complementing the attractor states known as goals, redirecting the flow of behavior to maximize the chances of survival and of reaching the actual goals.
As to the latter part of that paragraph, I think policy-selection for single goals and goal-selection more generally are important for enabling systems to exhibit flexible behavior. Someone in a recent thread (https://www.lesswrong.com/posts/3L46WGauGpr7nYubu/the-plan?commentId=nLCpJnxfaPzKXsbE2) brought up some interesting research on goal selection (more like goal pruning) in animals that could be worth looking into.