Intelligence solves problems, by guiding behavior to produce local extropy. It is indicated by the avoidance of probable outcomes, which is equivalent to the construction of information.
This amounts to something similar to the convergent instrumental goal definition; achieving sufficiently specific outcomes involves pursuing convergent instrumental goals.
I like the idea of looking for convergent instrumental goals, but I think this section specifically misses the opportunity to formalize the local extropy production or generally to look for information-theoretical measures.
If we assume a modeling of an agent in terms of his Markov blanket (ignoring issues with that for now[1]), then we could define the generalized capability of an agent in terms of that.
Capability=Ipred+Ictrl−βH(I)−S
Where
Ipred – “bits you can see coming”: The mutual information I(It;St+1) between the agent’s internal state It and its next sensory state St+1 quantifies how much the agent’s current “belief state” predicts what it will sense next.
Ictrl – “bits you can steer”: The mutual information I(At;Et+1) between the agent’s action At and the next external state Et+1 measures how much the agent’s outputs causally structure the world beyond its blanket.
H(I) – “bits you have to keep alive”: Shannon entropy of the internal state It. This is the size of the agent’s memory in bits. The coefficient β turns that size into a cost, reflecting physical maintenance energy and complexity overhead (e.g. Landauer limit).
S – “bits you fail to see coming”: Expected negative log-likelihood S=E[−logP(St+1∣It)] of the next sensory state given the internal state. This is the “leftover unpredictability” after using the best model encoded in It, i.e. the sensory free energy.
I like the idea of looking for convergent instrumental goals, but I think this section specifically misses the opportunity to formalize the local extropy production or generally to look for information-theoretical measures.
If we assume a modeling of an agent in terms of his Markov blanket (ignoring issues with that for now[1]), then we could define the generalized capability of an agent in terms of that.
Capability=Ipred+Ictrl−βH(I)−S
Where
Ipred – “bits you can see coming”:
The mutual information I(It;St+1) between the agent’s internal state It and its next sensory state St+1 quantifies how much the agent’s current “belief state” predicts what it will sense next.
Ictrl – “bits you can steer”:
The mutual information I(At;Et+1) between the agent’s action At and the next external state Et+1 measures how much the agent’s outputs causally structure the world beyond its blanket.
H(I) – “bits you have to keep alive”:
Shannon entropy of the internal state It. This is the size of the agent’s memory in bits. The coefficient β turns that size into a cost, reflecting physical maintenance energy and complexity overhead (e.g. Landauer limit).
S – “bits you fail to see coming”:
Expected negative log-likelihood S=E[−logP(St+1∣It)] of the next sensory state given the internal state. This is the “leftover unpredictability” after using the best model encoded in It, i.e. the sensory free energy.
Instead of the hard causal independence, it may be possible to define a boundary as the maximal separation in mutual information between clusters.