My aim is to work towards a paradigm where allying under a coherent identity is a demonstrably “reasonable” development for sub-agents or TIs subject to selection pressures. I agree that one of these pressures is likely predictability or, almost equivalently, a compressable and simple self-model.
I hint at what this could look like in the section “Robust equilibria and updatelessness”. The shape of identity (e.g. how updateless the agent allows itself to be) may dictate not only how TIs interact with each other, but also how they will navigate strategic interactions with TIs belonging to other agents. This would ideally mean that a theory of intrapersonal games in an agent determines what solution concepts are valid for that agent in usual game theory.
This addresses your point in the specific case where the rest of the world consists of other agents; I’m not as clear on the relationship intrapersonal games have to “world models”. Food for thought, thanks.
My aim is to work towards a paradigm where allying under a coherent identity is a demonstrably “reasonable” development for sub-agents or TIs subject to selection pressures. I agree that one of these pressures is likely predictability or, almost equivalently, a compressable and simple self-model.
I hint at what this could look like in the section “Robust equilibria and updatelessness”. The shape of identity (e.g. how updateless the agent allows itself to be) may dictate not only how TIs interact with each other, but also how they will navigate strategic interactions with TIs belonging to other agents. This would ideally mean that a theory of intrapersonal games in an agent determines what solution concepts are valid for that agent in usual game theory.
This addresses your point in the specific case where the rest of the world consists of other agents; I’m not as clear on the relationship intrapersonal games have to “world models”. Food for thought, thanks.