Thanks! Your definition of unbounded does not mention being unbounded below; that seems oddly asymmetrical to me.
I was wrong on this and Tim Tyler was right. The theorem takes an absolute value of U at the end, so apparently it’s still true if U has an unbounded absolute value. My guess that “unbounded” meant “unbounded below” was wrong, but apparently harmless.
I am still puzzled over what the environment functions are. f is one of the same kinds of things that h is, and h is “the true function that describes the environment”.
The f in Lemma 1 is a different sort of thing from the f in Equation 2 and Theorem 1. I assume you’re talking about Equation 2 and Theorem 1.
(Variable scope is ambiguous in mathematical texts and has to be guessed from context. Maybe I’m not stuck on this right now and you’re feeling stuck because I worked through Hutter’s AIXI paper. I spent a lot of time feeling stuck when reading that. Perhaps we all have to pay the same entry price. One might also argue that the papers are too concisely written.)
Our heroic AI is attempting to take action to maximize its expected utility. The only action it can take is to choose an input to h; suppose that’s i. The only response from the world is h(i). The utility it experiences is U(h(i)). There is no persistent state, and h never changes. It doesn’t know the true h, so the best it can do is maximize the expected value of U(f(i)), where f is a version of h that’s consistent with it’s observations. The weights in the expected value are p(f), which is the AI’s prior probability of f being h. The point of the paper is that it can’t evaluate that expected value because the series diverges when the U is unbounded.
Maybe we should do more math on this blog. It’s much more pleasant than arguing about philosophy. I would enjoy this conversation with Phil even if he was one of those goddamned moral realists. :-).
Knowing that this discussion exists here has increased the probability that I will actually attempt to work through the paper at some future time. Thanks.
At one point I was wondering whether I should talk about this via PM with Phil, or as replies to his posts. Since the level of interest in public replies was positive, I think I did the right thing. We should do more of this.
I was wrong on this and Tim Tyler was right. The theorem takes an absolute value of U at the end, so apparently it’s still true if U has an unbounded absolute value. My guess that “unbounded” meant “unbounded below” was wrong, but apparently harmless.
The f in Lemma 1 is a different sort of thing from the f in Equation 2 and Theorem 1. I assume you’re talking about Equation 2 and Theorem 1.
(Variable scope is ambiguous in mathematical texts and has to be guessed from context. Maybe I’m not stuck on this right now and you’re feeling stuck because I worked through Hutter’s AIXI paper. I spent a lot of time feeling stuck when reading that. Perhaps we all have to pay the same entry price. One might also argue that the papers are too concisely written.)
Our heroic AI is attempting to take action to maximize its expected utility. The only action it can take is to choose an input to h; suppose that’s i. The only response from the world is h(i). The utility it experiences is U(h(i)). There is no persistent state, and h never changes. It doesn’t know the true h, so the best it can do is maximize the expected value of U(f(i)), where f is a version of h that’s consistent with it’s observations. The weights in the expected value are p(f), which is the AI’s prior probability of f being h. The point of the paper is that it can’t evaluate that expected value because the series diverges when the U is unbounded.
Maybe we should do more math on this blog. It’s much more pleasant than arguing about philosophy. I would enjoy this conversation with Phil even if he was one of those goddamned moral realists. :-).
Knowing that this discussion exists here has increased the probability that I will actually attempt to work through the paper at some future time. Thanks.
At one point I was wondering whether I should talk about this via PM with Phil, or as replies to his posts. Since the level of interest in public replies was positive, I think I did the right thing. We should do more of this.