I don’t have my head around the proof in the paper, but I can answer your specific questions:
First, the paper needs to define what an unbounded function is.
U maps N to R and is unbounded, which means for all x in R there exists a y in N such that U(y) > x.
Next, in Lemma one that B(x) > f(x) infinitely often. But f(x) is defined as “a function that describes the environment”. There is no ordering defined on f(x); no interpretation of what the number you get out of f(x) means.
His point is that B grows really fast, faster than any computable f. I don’t see where it defines f(x) to be “a function that describes the environment”, and that definition wouldn’t make sense at this point anyway. Once he’s done proving B grows faster than any computable function, he doesn’t need that f any more.
Thanks! Your definition of unbounded does not mention being unbounded below; that seems oddly asymmetrical to me.
I am still puzzled over what the environment functions are. f is one of the same kinds of things that h is, and h is “the true function that describes the environment”.
Thanks! Your definition of unbounded does not mention being unbounded below; that seems oddly asymmetrical to me.
I was wrong on this and Tim Tyler was right. The theorem takes an absolute value of U at the end, so apparently it’s still true if U has an unbounded absolute value. My guess that “unbounded” meant “unbounded below” was wrong, but apparently harmless.
I am still puzzled over what the environment functions are. f is one of the same kinds of things that h is, and h is “the true function that describes the environment”.
The f in Lemma 1 is a different sort of thing from the f in Equation 2 and Theorem 1. I assume you’re talking about Equation 2 and Theorem 1.
(Variable scope is ambiguous in mathematical texts and has to be guessed from context. Maybe I’m not stuck on this right now and you’re feeling stuck because I worked through Hutter’s AIXI paper. I spent a lot of time feeling stuck when reading that. Perhaps we all have to pay the same entry price. One might also argue that the papers are too concisely written.)
Our heroic AI is attempting to take action to maximize its expected utility. The only action it can take is to choose an input to h; suppose that’s i. The only response from the world is h(i). The utility it experiences is U(h(i)). There is no persistent state, and h never changes. It doesn’t know the true h, so the best it can do is maximize the expected value of U(f(i)), where f is a version of h that’s consistent with it’s observations. The weights in the expected value are p(f), which is the AI’s prior probability of f being h. The point of the paper is that it can’t evaluate that expected value because the series diverges when the U is unbounded.
Maybe we should do more math on this blog. It’s much more pleasant than arguing about philosophy. I would enjoy this conversation with Phil even if he was one of those goddamned moral realists. :-).
Knowing that this discussion exists here has increased the probability that I will actually attempt to work through the paper at some future time. Thanks.
At one point I was wondering whether I should talk about this via PM with Phil, or as replies to his posts. Since the level of interest in public replies was positive, I think I did the right thing. We should do more of this.
I don’t have my head around the proof in the paper, but I can answer your specific questions:
U maps N to R and is unbounded, which means for all x in R there exists a y in N such that U(y) > x.
His point is that B grows really fast, faster than any computable f. I don’t see where it defines f(x) to be “a function that describes the environment”, and that definition wouldn’t make sense at this point anyway. Once he’s done proving B grows faster than any computable function, he doesn’t need that f any more.
Thanks! Your definition of unbounded does not mention being unbounded below; that seems oddly asymmetrical to me.
I am still puzzled over what the environment functions are. f is one of the same kinds of things that h is, and h is “the true function that describes the environment”.
Technically you should use the abs. value of the function, see here.
I was wrong on this and Tim Tyler was right. The theorem takes an absolute value of U at the end, so apparently it’s still true if U has an unbounded absolute value. My guess that “unbounded” meant “unbounded below” was wrong, but apparently harmless.
The f in Lemma 1 is a different sort of thing from the f in Equation 2 and Theorem 1. I assume you’re talking about Equation 2 and Theorem 1.
(Variable scope is ambiguous in mathematical texts and has to be guessed from context. Maybe I’m not stuck on this right now and you’re feeling stuck because I worked through Hutter’s AIXI paper. I spent a lot of time feeling stuck when reading that. Perhaps we all have to pay the same entry price. One might also argue that the papers are too concisely written.)
Our heroic AI is attempting to take action to maximize its expected utility. The only action it can take is to choose an input to h; suppose that’s i. The only response from the world is h(i). The utility it experiences is U(h(i)). There is no persistent state, and h never changes. It doesn’t know the true h, so the best it can do is maximize the expected value of U(f(i)), where f is a version of h that’s consistent with it’s observations. The weights in the expected value are p(f), which is the AI’s prior probability of f being h. The point of the paper is that it can’t evaluate that expected value because the series diverges when the U is unbounded.
Maybe we should do more math on this blog. It’s much more pleasant than arguing about philosophy. I would enjoy this conversation with Phil even if he was one of those goddamned moral realists. :-).
Knowing that this discussion exists here has increased the probability that I will actually attempt to work through the paper at some future time. Thanks.
At one point I was wondering whether I should talk about this via PM with Phil, or as replies to his posts. Since the level of interest in public replies was positive, I think I did the right thing. We should do more of this.