People who read this now should know that John and David have changed their minds! See https://www.lesswrong.com/posts/bzmLC3J8PsknwRZbr/why-not-subagents. The basic argument: Toad (mushroom lover) and the crab guy from futurama (anchovy lover) should be able to make binding commitments (possibly via acausal decision theory, or just reputation) allowing them to take the trades leaving them strictly better off as long as they even it out to be “fair” in the longer run (e.g. by finding a side channel to give “money” to even out the utilities gained, or by committing to allow the other agent to do trades more preferable to them in the future).
People who read this now should know that John and David have changed their minds! See https://www.lesswrong.com/posts/bzmLC3J8PsknwRZbr/why-not-subagents. The basic argument: Toad (mushroom lover) and the crab guy from futurama (anchovy lover) should be able to make binding commitments (possibly via acausal decision theory, or just reputation) allowing them to take the trades leaving them strictly better off as long as they even it out to be “fair” in the longer run (e.g. by finding a side channel to give “money” to even out the utilities gained, or by committing to allow the other agent to do trades more preferable to them in the future).