I still tentatively think the lexical update works, but it’s been a while and I might be missing something.
I’ll follow your convention that our universe is U-simple, our universal prior is U’, and so the aliens’ universe is U’-simple (I think—sorry if I got confused and that’s not what you mean).
If we sample from a universe that is U’-simple, then:
Assume the aliens care about U’-simplicity. They will preferentially sample from U’, and so have U’(our world) mass on our world. Within that, they will correctly guess that the machine they are supposed to control is using U’ as its prior. That is, they basically pay U’(our world) * P(us|someone using U’ to predict).
But our universal prior was also U’, wasn’t it? So we are also paying U’(our world) to pick out our world. I.e. we pay U’(our world) * P(someone making important predictions | our world) * P(someone using U’ to predict | someone making important predictions) * P(us|someone using U’ to predict).
I don’t see any program whose behavior depends on U(world) for the “real” simplicity prior U according to which our world is simple (and that concept seems slippery).
Thanks, that makes sense. Here is my rephrasing of the argument:
Let the ‘importance function’ Q(X,Y) take as inputs machines Xand Y, and output all places where Y is being used as a universal prior, weighted by their effect on X-short programs. Suppose for the sake of argument that there is some short program computing Q; this is probably the most ‘natural’ program of this form that we could hope for.
Even given such a program, we’ll still lose to the aliens: in U′, directly specifying our important decisions on Earth using Q will require both U and U′ to be fed into Q, costing KU′(U)+KU′(U′) bits, then KU′(us|Q(U,U′)) bits to specify us. For the aliens, getting them to be motivated to control U-short programs costs KU′(U) bits, but then they can skip directly to specifying us given Q(U,U′), so they save KU′(U′) bits over the direct explanation. So the lexical update works.
(I went wrong in thinking that the aliens would need to both update their notion of importance to match ours *and* locate our world; but if we assume the ‘importance function’ exists then the aliens can just pick out our world using our notion of importance)
I still tentatively think the lexical update works, but it’s been a while and I might be missing something.
I’ll follow your convention that our universe is U-simple, our universal prior is U’, and so the aliens’ universe is U’-simple (I think—sorry if I got confused and that’s not what you mean).
If we sample from a universe that is U’-simple, then:
Assume the aliens care about U’-simplicity. They will preferentially sample from U’, and so have U’(our world) mass on our world. Within that, they will correctly guess that the machine they are supposed to control is using U’ as its prior. That is, they basically pay U’(our world) * P(us|someone using U’ to predict).
But our universal prior was also U’, wasn’t it? So we are also paying U’(our world) to pick out our world. I.e. we pay U’(our world) * P(someone making important predictions | our world) * P(someone using U’ to predict | someone making important predictions) * P(us|someone using U’ to predict).
I don’t see any program whose behavior depends on U(world) for the “real” simplicity prior U according to which our world is simple (and that concept seems slippery).
Does that seem right?
Thanks, that makes sense. Here is my rephrasing of the argument:
Let the ‘importance function’ Q(X,Y) take as inputs machines Xand Y, and output all places where Y is being used as a universal prior, weighted by their effect on X-short programs. Suppose for the sake of argument that there is some short program computing Q; this is probably the most ‘natural’ program of this form that we could hope for.
Even given such a program, we’ll still lose to the aliens: in U′, directly specifying our important decisions on Earth using Q will require both U and U′ to be fed into Q, costing KU′(U)+KU′(U′) bits, then KU′(us|Q(U,U′)) bits to specify us. For the aliens, getting them to be motivated to control U-short programs costs KU′(U) bits, but then they can skip directly to specifying us given Q(U,U′), so they save KU′(U′) bits over the direct explanation. So the lexical update works.
(I went wrong in thinking that the aliens would need to both update their notion of importance to match ours *and* locate our world; but if we assume the ‘importance function’ exists then the aliens can just pick out our world using our notion of importance)