Ok, I think in that case my argument doesn’t work. Let me try another approach.
Suppose some stranger appears to you and says that you’re living in a simulated world. Out in the real world there is another simulation that contains 3^^^^3 identical copies of a utopian Earth-like planet plus another 3^^^^3 identical copies of a less utopian (but still pretty good) planet.
Now, if you press this button, you’ll turn X of the utopian planets into copies of the less utopian planet, where X is a 10^100 digit random number. (Note that K(X) is of order 10^100 which is much larger than K(3^^^^3) and so pressing the button would increase the Kolmogorov complexity of that simulated world by about 10^100.)
What does your proposed utility function say you should do (how much would you pay to either press the button or prevent it being pressed), and why?
Utility is monotonic, even though complexity isn’t. (Thus X downgrades out of the 3^^^^3 wouldn’t be as bad as, say, 3^^^3 downgrades.) However, utility is bounded by complexity: the complexity of a scenario with utility N must be at least N. (Asymptotically, of course.)
Ok, I think in that case my argument doesn’t work. Let me try another approach.
Suppose some stranger appears to you and says that you’re living in a simulated world. Out in the real world there is another simulation that contains 3^^^^3 identical copies of a utopian Earth-like planet plus another 3^^^^3 identical copies of a less utopian (but still pretty good) planet.
Now, if you press this button, you’ll turn X of the utopian planets into copies of the less utopian planet, where X is a 10^100 digit random number. (Note that K(X) is of order 10^100 which is much larger than K(3^^^^3) and so pressing the button would increase the Kolmogorov complexity of that simulated world by about 10^100.)
What does your proposed utility function say you should do (how much would you pay to either press the button or prevent it being pressed), and why?
Utility is monotonic, even though complexity isn’t. (Thus X downgrades out of the 3^^^^3 wouldn’t be as bad as, say, 3^^^3 downgrades.) However, utility is bounded by complexity: the complexity of a scenario with utility N must be at least N. (Asymptotically, of course.)