If we were to try to go updateless “all the way” and unupdate on a ton of information we’ve already learned, this could have pretty intense implications for ECL. For example, we might no longer have any particular reason to believe that our own values are common across the universe[1] (insofar as our beliefs about this are largely based on our observations), so when deciding what values to optimize for, we might give more weight to other values.
And the case for updatelessness is pretty strong if you endorse EDT, given that EDT with updatelessness double counts. I think this is especially true for empirical updatelessness, but also somewhat true for logical updatelessness. (See my comments on that post.)
Okay, so I think you’re right that strong updatelessness like that can change the implications of ECL (although I guess I didn’t technically say otherwise, I only said that updatelessness isn’t needed to get to a version of ECL—and of course, commitment theory isn’t my actual position anyway).
You could see my post that I linked above as an argument in a direction similar to Christiano’s point too—that the possibility of being a freak observer makes you revert back to your ~prior over and over again with every decision (since all your memories could be fake). It’s kind of the very elaborated/general (see especially footnotes 10 and 11) version of this brief digression in one of your comments: “In large worlds where your observations have sufficient randomness that observers of all kinds exists in all worlds, the SSA update step cannot exclude any world. You’re updateless by default.”
However, my current general attitude on a lot of this stuff is that I don’t understand why the decision-theoretic perspective of “how updateless should we be” is a better framing than the metaphysical/ethical perspective of “what should we think of as existing/mattering”. In small worlds with no copies of you (i. e. rejecting one of Christiano’s assumptions), EDT with updating intuitively does fine (or at least doesn’t double count). So the crux here feels less productively described as “how updateless to be” than as our metaphysics/ethics.
For example: “If we go updateless all the way, we might no longer have any particular reason to believe that our own values are common across the universe”. This feels like a confused way to point at the underlying concern (of how to sum up / weigh all the possibilities, which I share) - and confused precisely because it avoids the metaphysical implications. It would be more accurate to say that if we go updateless all the way, we don’t even know what kind of universe we’re in in the first place, if it’s even large and whether ECL is justified at all, etc. (footnote 11 might be relevant here again). If you’re restricting yourself to “the universe as we know it”, you’re not really going “updateless all the way”—so of course this will lead to absurdity (like the conclusion that we can’t get any information about this universe based on our observations in it). (More accurately, it’s selectively updateless—updateless about the existence of our entire civilization, our values, decision processes, etc—but not about the information of our universe being very large that we got from our human community of physicists. That seems nonsensical.)
What do you think? I’m pretty unsure here, definitely still feel confused. Feel free to only respond briefly or to part of it. (also, I left a brief commenton the Christiano post if you’re interested).
I think updatelessness can be important for ECL.
If we were to try to go updateless “all the way” and unupdate on a ton of information we’ve already learned, this could have pretty intense implications for ECL. For example, we might no longer have any particular reason to believe that our own values are common across the universe[1] (insofar as our beliefs about this are largely based on our observations), so when deciding what values to optimize for, we might give more weight to other values.
And the case for updatelessness is pretty strong if you endorse EDT, given that EDT with updatelessness double counts. I think this is especially true for empirical updatelessness, but also somewhat true for logical updatelessness. (See my comments on that post.)
Or that our values are especially correlated with our decision algorithms, which matters a lot for ECL.
Thanks for finally making me read that Paul Christiano post in depth—extremely interesting. I’ll need to digest it for a while.
(until then, you may be interested in my most recent post, where I try to give a metaphysical justification for something like updatelessness)
Okay, so I think you’re right that strong updatelessness like that can change the implications of ECL (although I guess I didn’t technically say otherwise, I only said that updatelessness isn’t needed to get to a version of ECL—and of course, commitment theory isn’t my actual position anyway).
You could see my post that I linked above as an argument in a direction similar to Christiano’s point too—that the possibility of being a freak observer makes you revert back to your ~prior over and over again with every decision (since all your memories could be fake). It’s kind of the very elaborated/general (see especially footnotes 10 and 11) version of this brief digression in one of your comments: “In large worlds where your observations have sufficient randomness that observers of all kinds exists in all worlds, the SSA update step cannot exclude any world. You’re updateless by default.”
However, my current general attitude on a lot of this stuff is that I don’t understand why the decision-theoretic perspective of “how updateless should we be” is a better framing than the metaphysical/ethical perspective of “what should we think of as existing/mattering”. In small worlds with no copies of you (i. e. rejecting one of Christiano’s assumptions), EDT with updating intuitively does fine (or at least doesn’t double count). So the crux here feels less productively described as “how updateless to be” than as our metaphysics/ethics.
For example: “If we go updateless all the way, we might no longer have any particular reason to believe that our own values are common across the universe”. This feels like a confused way to point at the underlying concern (of how to sum up / weigh all the possibilities, which I share) - and confused precisely because it avoids the metaphysical implications. It would be more accurate to say that if we go updateless all the way, we don’t even know what kind of universe we’re in in the first place, if it’s even large and whether ECL is justified at all, etc. (footnote 11 might be relevant here again). If you’re restricting yourself to “the universe as we know it”, you’re not really going “updateless all the way”—so of course this will lead to absurdity (like the conclusion that we can’t get any information about this universe based on our observations in it). (More accurately, it’s selectively updateless—updateless about the existence of our entire civilization, our values, decision processes, etc—but not about the information of our universe being very large that we got from our human community of physicists. That seems nonsensical.)
What do you think? I’m pretty unsure here, definitely still feel confused. Feel free to only respond briefly or to part of it. (also, I left a brief comment on the Christiano post if you’re interested).
(sorry for the nitpick but I think you meant to say EDT with updatefulness double counts?)