The interesting part is how systems/pre-agents/egregores/whatever become complete.
If it already satisfies the other VNM axioms we can analyse the situation as follows:
Recall that ain inexploitable but incomplete VNM agents acts like a Vetocracy of VNM agents. The exact decomposition is underspecified by just the preference order and is another piece of data (hidden state).
However, given sure-gain offers from the environment there is selection pressure for the internal complete VNM Subagents to make trade agreements to obtain a pareto improvement.
If you analyze this it looks like a simple prisoner dilemma type case which can be analyzed the usual way in game theory. For instance, in repeated offers with uncertain horizon the Subagents may be able to cooperate.
Once they are (approximately) complete they will be under selection pressure to satisfy the other axioms. You could say this the beginning of ‘emergence of expected utility maximizers’
As you can see the key here is that we really should be talking about Selection Theorems not the highly simplified Coherence Theorems. Coherence theorems are about ideal agents.
Selection theorems are about how more and more coherent and goal-directed agents may emerge.
Agree.
There are three stages:
Selection for inexploitability
The interesting part is how systems/pre-agents/egregores/whatever become complete.
If it already satisfies the other VNM axioms we can analyse the situation as follows: Recall that ain inexploitable but incomplete VNM agents acts like a Vetocracy of VNM agents. The exact decomposition is underspecified by just the preference order and is another piece of data (hidden state). However, given sure-gain offers from the environment there is selection pressure for the internal complete VNM Subagents to make trade agreements to obtain a pareto improvement. If you analyze this it looks like a simple prisoner dilemma type case which can be analyzed the usual way in game theory. For instance, in repeated offers with uncertain horizon the Subagents may be able to cooperate.
Once they are (approximately) complete they will be under selection pressure to satisfy the other axioms. You could say this the beginning of ‘emergence of expected utility maximizers’
As you can see the key here is that we really should be talking about Selection Theorems not the highly simplified Coherence Theorems. Coherence theorems are about ideal agents. Selection theorems are about how more and more coherent and goal-directed agents may emerge.