It doesn’t model its past self as a goal seeking agent,
Can you justify this claim more? If a goal seeking agent is a good model for the past behavior of the part of the universe where AIXI is functioning won’t it adopt that model with a high probability? It might not understand in some sense that it is the entity in question, but a hypothesis resembling “there’s an entity which tries to maximizes or minimize grues depending on the state of this toggle switch” will once it wants to maximize or minimize cause the AIXI to protect the switch’s current status. I’m not certain of this; there may be something I’m missing here.
Typo.
Can you justify this claim more? If a goal seeking agent is a good model for the past behavior of the part of the universe where AIXI is functioning won’t it adopt that model with a high probability? It might not understand in some sense that it is the entity in question, but a hypothesis resembling “there’s an entity which tries to maximizes or minimize grues depending on the state of this toggle switch” will once it wants to maximize or minimize cause the AIXI to protect the switch’s current status. I’m not certain of this; there may be something I’m missing here.