It is these models which will determine A’s behavior, and the result is almost certainly very arbitrary (this is similar to some discussions of Pascal’s Wager: it is possible that all of these strange models will cancel out and add up to normality, but it seems outlandishly unlikely).
I don’t understand.
Suppose we have two models, 3.1 and 3.2. 3.1 is: “Given an output sequence n, add a to utility gained.” 3.2 is: “Given an output sequence n, subtract a from utility gained.”
It looks to me like these should have the same complexity. In addition, if AIXI outputs the sequence n, and no change to utility is observed, and 3.1 and 3.2 have the same prior probability, then they should wind up with the same posterior probability. Since AIXI is considering every possible model, for any model of the form 3.1, it will also consider a model of the form 3.2, and the two should cancel out. Since all of these pairs of models should cancel out, the class of Model 3-like models should have as little impact on the final action as Model 2.
Or you saying that the failure of Model 3-like models to sum to zero is a consequence of the approximations in AIXItl?
I don’t understand.
Suppose we have two models, 3.1 and 3.2. 3.1 is: “Given an output sequence n, add a to utility gained.” 3.2 is: “Given an output sequence n, subtract a from utility gained.”
It looks to me like these should have the same complexity. In addition, if AIXI outputs the sequence n, and no change to utility is observed, and 3.1 and 3.2 have the same prior probability, then they should wind up with the same posterior probability. Since AIXI is considering every possible model, for any model of the form 3.1, it will also consider a model of the form 3.2, and the two should cancel out. Since all of these pairs of models should cancel out, the class of Model 3-like models should have as little impact on the final action as Model 2.
Or you saying that the failure of Model 3-like models to sum to zero is a consequence of the approximations in AIXItl?