AIXI does expectimax to decide upon actions, and works under the assumption that after the current action, the next action will also be decided by expectimax. That’s built into the source code.
Now, maybe you could “fake” a change to this assumption with a world-program that throws away AIXI’s output channel, and substitutes the action that would be taken by the modified AIXI. Of course, since AIXI itself is uncomputable, for any nontrivial modification that is not just a transformation of AIXI’s output, but leaves AIXI still uncomputable, this program doesn’t exist.
For AIXItl you may have the same problem manifest in the form of a program that simulates AIXItl “taking too long”. Not sure about that.
Either way, it’s not enough for such world programs to “ever” accumulate enough probability mass to dominate AIXI’s predictions. They’d have to dominate AIXI’s predictions before any modification event has been observed to be of any use to AIXI in deciding how to self-modify.
I’m not trying to claim that AIXI is a good model in which to explore self-modification. My issue isn’t on the agent-y side at all—it’s on the learning side. It has been put forward that there are facts about the world that AIXI is incapable of learning, even though humans are quite capable of learning them. (I’m assuming here that the environment is sufficiently information-rich that these facts are within reach.) To be more specific, the claim is that humans can learn facts about the observable universe that Solomonoff induction can’t. To me, this claim seems to imply that human learning is not computable, and this implication makes my brain emit, “Error! Error! Does not compute!”
AIXI does expectimax to decide upon actions, and works under the assumption that after the current action, the next action will also be decided by expectimax. That’s built into the source code.
Now, maybe you could “fake” a change to this assumption with a world-program that throws away AIXI’s output channel, and substitutes the action that would be taken by the modified AIXI. Of course, since AIXI itself is uncomputable, for any nontrivial modification that is not just a transformation of AIXI’s output, but leaves AIXI still uncomputable, this program doesn’t exist.
For AIXItl you may have the same problem manifest in the form of a program that simulates AIXItl “taking too long”. Not sure about that.
Either way, it’s not enough for such world programs to “ever” accumulate enough probability mass to dominate AIXI’s predictions. They’d have to dominate AIXI’s predictions before any modification event has been observed to be of any use to AIXI in deciding how to self-modify.
It might be possible?
I’m not trying to claim that AIXI is a good model in which to explore self-modification. My issue isn’t on the agent-y side at all—it’s on the learning side. It has been put forward that there are facts about the world that AIXI is incapable of learning, even though humans are quite capable of learning them. (I’m assuming here that the environment is sufficiently information-rich that these facts are within reach.) To be more specific, the claim is that humans can learn facts about the observable universe that Solomonoff induction can’t. To me, this claim seems to imply that human learning is not computable, and this implication makes my brain emit, “Error! Error! Does not compute!”