Three separate comment threads simultaneously lead to the refutation that I seem to be unfairly biased towards agents that happen to be born in simple universes. I think this is a good point, so I am starting a new comment thread to discuss that issue. Here is my counter point.
First, notice that our finite intuitions do not follow over nicely here. In saying that beings in universes with 20 fewer bits are a million times as important, I am not saying that the happiness of this one person is more important than the happiness of those million people over there. Instead, I am pointing at two infinite and unmeasurable clusters of universes, and saying that this cluster is a million times as important as this other cluster. Because there is no measure on these clusters, there is no fact of the matter as to whether one cluster is a million times as large as another. In finite collections, you do not have this issue, but with infinite collections, there could million to one map from one cluster to another, and a million to one map in the other direction. To judge me as unfair, you must put a measure on the collection of universes by which to judge.
So, what measure should we put on the collection of universes to judge this fairness? It may look like my measure is unfair because it is not uniform, giving much more weight to simple universes. However, I argue that it is the most fair. If your collection of universes are described in some language on an infinite tape, I am giving a uniform distribution of weight over all infinite tapes. However, this means that universes with simple finite descriptions can ignore most of the tape and show up more in the uniform distribution over infinite tapes. What looks unfair to you is actually the uniform weighting in a very slightly different, (and perhaps more natural) model—the model that VAuroch argues for in his comments here.
I think that Solomonoff Induction already gets us to the conclusion we want, with the problem that it is relative to a language. So one way to put your point would be this: “There is no fact of the matter about which language is ‘right,’ what really exists is an infinite unordered jumble of universes. In order to think about the jumble, much less describe values over it, we must fix a language with which to describe it. Why not pick a language that favors a certain pleasing kind of simplicity? And hey, if we do this, then thanks to SI it all adds up to normality!”
Retreating into the impregnable swamp of infinity may save you here, but it is a dubious move in general. Compare to someone who thinks that they will win the lottery, because they believe in a Big World and thus there are infinitely many copies/futures that will win the lottery.
Edit: Thank you, by the way, for this conversation and discussion. I’m very interested in this topic and I like the way you’ve been thinking about it. I hope we can continue to make progress!
Thank you all so much for all of your comments.
Three separate comment threads simultaneously lead to the refutation that I seem to be unfairly biased towards agents that happen to be born in simple universes. I think this is a good point, so I am starting a new comment thread to discuss that issue. Here is my counter point.
First, notice that our finite intuitions do not follow over nicely here. In saying that beings in universes with 20 fewer bits are a million times as important, I am not saying that the happiness of this one person is more important than the happiness of those million people over there. Instead, I am pointing at two infinite and unmeasurable clusters of universes, and saying that this cluster is a million times as important as this other cluster. Because there is no measure on these clusters, there is no fact of the matter as to whether one cluster is a million times as large as another. In finite collections, you do not have this issue, but with infinite collections, there could million to one map from one cluster to another, and a million to one map in the other direction. To judge me as unfair, you must put a measure on the collection of universes by which to judge.
So, what measure should we put on the collection of universes to judge this fairness? It may look like my measure is unfair because it is not uniform, giving much more weight to simple universes. However, I argue that it is the most fair. If your collection of universes are described in some language on an infinite tape, I am giving a uniform distribution of weight over all infinite tapes. However, this means that universes with simple finite descriptions can ignore most of the tape and show up more in the uniform distribution over infinite tapes. What looks unfair to you is actually the uniform weighting in a very slightly different, (and perhaps more natural) model—the model that VAuroch argues for in his comments here.
I think that Solomonoff Induction already gets us to the conclusion we want, with the problem that it is relative to a language. So one way to put your point would be this: “There is no fact of the matter about which language is ‘right,’ what really exists is an infinite unordered jumble of universes. In order to think about the jumble, much less describe values over it, we must fix a language with which to describe it. Why not pick a language that favors a certain pleasing kind of simplicity? And hey, if we do this, then thanks to SI it all adds up to normality!”
Retreating into the impregnable swamp of infinity may save you here, but it is a dubious move in general. Compare to someone who thinks that they will win the lottery, because they believe in a Big World and thus there are infinitely many copies/futures that will win the lottery.
Edit: Thank you, by the way, for this conversation and discussion. I’m very interested in this topic and I like the way you’ve been thinking about it. I hope we can continue to make progress!