The set S is the set of all total recursive functions. This is set in stone for all time. Therefore, there is only one way that S_I can refer to different things:
Our stock of observational data may be different. In other words, the set I and the values of h(i) for i in I may be different.
But regardless of I and the values of h(i), it’s easy to see that one cannot restrict S_I in the way you’re attempting to do.
In fact, one can easily see that S_I = the set of functions of the form “if x is in I then h(x), otherwise f(x)” where f is an arbitrary recursive function.
That is, the whole “I” business is completely pointless, except (presumably) to help the reader assure themselves that the result does apply to AIXI.
So, we can rescue our utility function from the theorem if we are allowed to assign zero probability to arbitrary hypotheses that have no plausibility other than that they have not been absolutely ruled out. Such as the hypothesis that the laws of physics are valid at all times except on October 21, 2011.
Being allowed to do this would make the counterexample work.
The set S is the set of all total recursive functions. This is set in stone for all time. Therefore, there is only one way that S_I can refer to different things:
Our stock of observational data may be different. In other words, the set I and the values of h(i) for i in I may be different.
But regardless of I and the values of h(i), it’s easy to see that one cannot restrict S_I in the way you’re attempting to do.
In fact, one can easily see that S_I = the set of functions of the form “if x is in I then h(x), otherwise f(x)” where f is an arbitrary recursive function.
That is, the whole “I” business is completely pointless, except (presumably) to help the reader assure themselves that the result does apply to AIXI.
So, we can rescue our utility function from the theorem if we are allowed to assign zero probability to arbitrary hypotheses that have no plausibility other than that they have not been absolutely ruled out. Such as the hypothesis that the laws of physics are valid at all times except on October 21, 2011.
Being allowed to do this would make the counterexample work.