I’m really confused. Like, really, really confused. Hopefully someone can illuminate this topic for me, because right now, I’m not seeing where this “leverage penalty” comes from. Complexity penalties are pretty obviously a consequence of formalizations of Occam’s Razor, in particular Solomonoff Induction, but why does the idea of a “leverage penalty” even exist? It seems like a post hoc justification tacked on in order to somehow deal with the original Pascal’s Mugging situation. If I started from the basics of probability theory and computational theory, it seems conceivable to me that given enough time, I might be able to independently arrive at the idea of complexity penalties. It does not, on the other hand, seem likely that I would ever be able to derive this concept of a “leverage penalty” from first principles; it seems like a clever after-the-fact justification.
I do realize, however, that the leverage penalty was proposed by a very smart person (Robin Hanson), and then later discussed by another very smart person (Eliezer), both of whom are much smarter than I am, so it is much more likely that I am the one confused here than that they are actually engaging in after-the-fact rationalization. So my question right now is this: where do “leverage penalties” come from? Could someone take the time to humor an aspiring student of mathematics and explain? Thanks in advance!
(Right now, I’m not sure where leverage penalties come from, but if they do come from somewhere, as opposed to being pulled out of thin air, my bet is on anthropics. If this is true, it wouldn’t be surprising, because I find anthropics hellishly confusing most of the time, so it seems reasonable that I would be confused about a concept derived from that area.)
Anthropics would be one way of reading it, yes. Think of it as saying, in addition to wanting all of our Turing machines to add up to 1, we also want all of the computational elements inside our Turing machines to add up to 1 because we’re trying to guess which computational element ‘we’ might be. This might seem badly motivated in the sense that we can only say “Because our probabilities have to add up to 1 for us to think!” rather than being able to explain why magical reality fluid ought to work that way a priori, but the justification for a simplicity prior isn’t much different—we have to be able to add up all the Turing machines in their entirety to 1 in order to think. So Turing machines that use lots of tape get penalties to the probability of your being any particular or special element inside them. Being able to affect lots of other elements is a kind of specialness.
I’m confused because I had always thought it would be the exact opposite. To predict your observational history given a description of the universe, solomonoff induction needs to find you in it. The more special you are, the easier you are to find and thus the easier it is to find your observational history.
I’m really confused. Like, really, really confused. Hopefully someone can illuminate this topic for me, because right now, I’m not seeing where this “leverage penalty” comes from. Complexity penalties are pretty obviously a consequence of formalizations of Occam’s Razor, in particular Solomonoff Induction, but why does the idea of a “leverage penalty” even exist? It seems like a post hoc justification tacked on in order to somehow deal with the original Pascal’s Mugging situation. If I started from the basics of probability theory and computational theory, it seems conceivable to me that given enough time, I might be able to independently arrive at the idea of complexity penalties. It does not, on the other hand, seem likely that I would ever be able to derive this concept of a “leverage penalty” from first principles; it seems like a clever after-the-fact justification.
I do realize, however, that the leverage penalty was proposed by a very smart person (Robin Hanson), and then later discussed by another very smart person (Eliezer), both of whom are much smarter than I am, so it is much more likely that I am the one confused here than that they are actually engaging in after-the-fact rationalization. So my question right now is this: where do “leverage penalties” come from? Could someone take the time to humor an aspiring student of mathematics and explain? Thanks in advance!
(Right now, I’m not sure where leverage penalties come from, but if they do come from somewhere, as opposed to being pulled out of thin air, my bet is on anthropics. If this is true, it wouldn’t be surprising, because I find anthropics hellishly confusing most of the time, so it seems reasonable that I would be confused about a concept derived from that area.)
Anthropics would be one way of reading it, yes. Think of it as saying, in addition to wanting all of our Turing machines to add up to 1, we also want all of the computational elements inside our Turing machines to add up to 1 because we’re trying to guess which computational element ‘we’ might be. This might seem badly motivated in the sense that we can only say “Because our probabilities have to add up to 1 for us to think!” rather than being able to explain why magical reality fluid ought to work that way a priori, but the justification for a simplicity prior isn’t much different—we have to be able to add up all the Turing machines in their entirety to 1 in order to think. So Turing machines that use lots of tape get penalties to the probability of your being any particular or special element inside them. Being able to affect lots of other elements is a kind of specialness.
I’m confused because I had always thought it would be the exact opposite. To predict your observational history given a description of the universe, solomonoff induction needs to find you in it. The more special you are, the easier you are to find and thus the easier it is to find your observational history.