It’s not just that it doesn’t “have a simple definition in terms of fundamental physics”. Rather, it can’t be given any sense at all.
The thing is, Eliezer jumped directly from “physical continuity can’t be given sense in terms of persistent identity of fundamental particles” to “we should stop caring about physical continuity” while skipping the step of “physical continuity can’t be given any other sense”. How did he do this, except by relying implicitly on the assumption that it should “have a simple definition in terms of fundamental physics” or something similar, even as he would surely deny this assumption when considering it explicitly?
I would say that what I did is more like continuing to care about continuity, but trying to put it into causal continuity or pattern continuity after the particular hypothesis of ‘particle identity continuity’ turned out to be nonsense. Also, I regard this as a problem not strictly of utility functions because it controls what I expect to see happen after being cryonically revived or stepping into a Star Trek transporter—I either see the next moment, or see what happens after dying in a car crash i.e. NULL.
The thing is, Eliezer jumped directly from “physical continuity can’t be given sense in terms of persistent identity of fundamental particles” to “we should stop caring about physical continuity” while skipping the step of “physical continuity can’t be given any other sense”. How did he do this, except by relying implicitly on the assumption that it should “have a simple definition in terms of fundamental physics” or something similar, even as he would surely deny this assumption when considering it explicitly?
I would say that what I did is more like continuing to care about continuity, but trying to put it into causal continuity or pattern continuity after the particular hypothesis of ‘particle identity continuity’ turned out to be nonsense. Also, I regard this as a problem not strictly of utility functions because it controls what I expect to see happen after being cryonically revived or stepping into a Star Trek transporter—I either see the next moment, or see what happens after dying in a car crash i.e. NULL.