There’s a point by Stuart Armstrong that anthropic updates are non-Bayesian, because you can think of Bayesian updates as deprecating improbable hypotheses and renormalizing, while anthropic updates (e.g. updating on “I think just got copied”) require increasing probability on previously unlikely hypotheses.
In the last few years I’ve started thinking “what would a Solomonoff inductor do?” more often about anthropic questions. So I just thought about this case, and I realized there’s something interesting (to me at least).
Suppose we’re in the cloning version of Sleeping Beauty. So if the coin landed Heads, the sign outside the room will say Room A, if the coin landed Tails, the sign could say either Room A or Room B. Normally I translate this into Solomonoff-ese by treating different hypotheses about the world as Turing machines that could reproduce my memories. Each hypothesis is like a simulation of the world (with some random seed), plus some rule for specifying what physical features (i.e. my memories) to read to the output tape. So “Heads in Room A,” “Tails in Room A,” and “Tails in Room B” are three different Turing machines that reproduce the sequence of my memories by simulating a physical universe.
Where’s the non-Bayesian-ness? Well, it’s because before getting copied, but after the coin was flipped, there were supposed to only be two clumps of hypotheses—let’s call them “Heads and walking down the hallway” and “Tails and walking down the hallway.” And so before getting copied you assign only 50% to Tails. And so P(Tails|memories after getting copied) is made of two hypotheses while P(Tails|memories before getting copied) is only made of one hypothesis, so the extra hypothesis got bumped up non-Bayesianly.
But… hang on. Why can’t “Tails in Room B” get invoked as a second hypothesis for my memories even before getting copied? After all, it’s just as good at reproducing my past memories—the sequence just happens to continue on a bit.
What I think is happening here is that we have revealed a difference between actual Solomonoff induction, and my intuitive translation of hypotheses into Turing machines. In actual (or at least typical) Solomonoff induction, it’s totally normal if the sequence happens to continue! But in the hypotheses that correspond to worlds at specific points in time, the Turing machine reads out my current memories and only my current memories. At first blush, this seems to me to be an arbitrary choice—is there some reason why it’s non-arbitrary?
We could appeal to the imperfection of human memory (so that only current-me has my exact memories), but I don’t like the sound of that, because anthropics doesn’t seem like it should undergo a discontinuous transition if we get better memory.
Do you have a link to that argument? I think Bayesean updates include either reducing a prior or increasing it, and then renormalizing all related probabilities. Many updatable observations take the form of replacing an estimate of future experience (I will observe sunshine tomorrow) by a 1 or zero (I did or did not observe that, possibly not quite 0 or 1 if you want to account for hallucinations and imperfect memory).
Anthropic updates are either bayesean or impossible. The underlying question remains “how does this experience differ from my probability estimate”? For Bayes or for Solomonoff, one has to answer “what has changed for my prediction? In what way am I surprised and have to change my calculation?”
I have a totally non-Solomonoff explanation of what’s going on, which actually goes full G.K. Chesterton—I assign anthropic probabilities because I don’t assume that my not waking is impossible. But I’m not sure how a Solomonoff inductor would see it.
There’s a point by Stuart Armstrong that anthropic updates are non-Bayesian, because you can think of Bayesian updates as deprecating improbable hypotheses and renormalizing, while anthropic updates (e.g. updating on “I think just got copied”) require increasing probability on previously unlikely hypotheses.
In the last few years I’ve started thinking “what would a Solomonoff inductor do?” more often about anthropic questions. So I just thought about this case, and I realized there’s something interesting (to me at least).
Suppose we’re in the cloning version of Sleeping Beauty. So if the coin landed Heads, the sign outside the room will say Room A, if the coin landed Tails, the sign could say either Room A or Room B. Normally I translate this into Solomonoff-ese by treating different hypotheses about the world as Turing machines that could reproduce my memories. Each hypothesis is like a simulation of the world (with some random seed), plus some rule for specifying what physical features (i.e. my memories) to read to the output tape. So “Heads in Room A,” “Tails in Room A,” and “Tails in Room B” are three different Turing machines that reproduce the sequence of my memories by simulating a physical universe.
Where’s the non-Bayesian-ness? Well, it’s because before getting copied, but after the coin was flipped, there were supposed to only be two clumps of hypotheses—let’s call them “Heads and walking down the hallway” and “Tails and walking down the hallway.” And so before getting copied you assign only 50% to Tails. And so P(Tails|memories after getting copied) is made of two hypotheses while P(Tails|memories before getting copied) is only made of one hypothesis, so the extra hypothesis got bumped up non-Bayesianly.
But… hang on. Why can’t “Tails in Room B” get invoked as a second hypothesis for my memories even before getting copied? After all, it’s just as good at reproducing my past memories—the sequence just happens to continue on a bit.
What I think is happening here is that we have revealed a difference between actual Solomonoff induction, and my intuitive translation of hypotheses into Turing machines. In actual (or at least typical) Solomonoff induction, it’s totally normal if the sequence happens to continue! But in the hypotheses that correspond to worlds at specific points in time, the Turing machine reads out my current memories and only my current memories. At first blush, this seems to me to be an arbitrary choice—is there some reason why it’s non-arbitrary?
We could appeal to the imperfection of human memory (so that only current-me has my exact memories), but I don’t like the sound of that, because anthropics doesn’t seem like it should undergo a discontinuous transition if we get better memory.
Do you have a link to that argument? I think Bayesean updates include either reducing a prior or increasing it, and then renormalizing all related probabilities. Many updatable observations take the form of replacing an estimate of future experience (I will observe sunshine tomorrow) by a 1 or zero (I did or did not observe that, possibly not quite 0 or 1 if you want to account for hallucinations and imperfect memory).
Anthropic updates are either bayesean or impossible. The underlying question remains “how does this experience differ from my probability estimate”? For Bayes or for Solomonoff, one has to answer “what has changed for my prediction? In what way am I surprised and have to change my calculation?”
https://www.alignmentforum.org/posts/iNi8bSYexYGn9kiRh/paradoxes-in-all-anthropic-probabilities I think?
I have a totally non-Solomonoff explanation of what’s going on, which actually goes full G.K. Chesterton—I assign anthropic probabilities because I don’t assume that my not waking is impossible. But I’m not sure how a Solomonoff inductor would see it.