Is this sort of a way to get an agent with a DT that admits acausal trade (as we think the correct decision theory would) to act more like a CDT agent? I wonder how different the behaviors of the agent you specify are from those of a CDT agent—in what kinds of situations would they come apart? When does “I only value what happens given that I exist” (roughly) differ from “I only value what I directly cause” (roughly)?
I am concerned about modeling nonexistence as zero or infinitely negative utility. That sort of thing leads to disincentivizing the utility function in circumstances where death is likely. Harry in HPMOR, for example, doesn’t want his parents to be tortured regardless of whether he’s dead, such that he is willing to take on an increased risk of death to ensure that such will not happen, and I think the same invariance should hold true for FAI. That is not to say that it should be susceptible to blackmail; Harry ensured his parents’ safety with a decidedly detrimental effect on his opponents.
Is this sort of a way to get an agent with a DT that admits acausal trade (as we think the correct decision theory would) to act more like a CDT agent? I wonder how different the behaviors of the agent you specify are from those of a CDT agent—in what kinds of situations would they come apart? When does “I only value what happens given that I exist” (roughly) differ from “I only value what I directly cause” (roughly)?
I am concerned about modeling nonexistence as zero or infinitely negative utility. That sort of thing leads to disincentivizing the utility function in circumstances where death is likely. Harry in HPMOR, for example, doesn’t want his parents to be tortured regardless of whether he’s dead, such that he is willing to take on an increased risk of death to ensure that such will not happen, and I think the same invariance should hold true for FAI. That is not to say that it should be susceptible to blackmail; Harry ensured his parents’ safety with a decidedly detrimental effect on his opponents.
Acausal trade with agents who can check whether you exist or not.
Can those agents check whether your utility function is p vs p’? Because otherwise the point seems moot.
They can have a probability estimate over it. Just as in all acausal trade. Which I don’t fully understand.
CDT is not stable, and we’re not sure where that decision theory could end up at.
It seems this approach could be plugged into even a stable decision theory.
Or, more interestingly, we might be able to turn on certain acausal trades and turn off others.