A.Crossman:
Prase, Chris, I don’t understand. Eliezer’s example is set up in such a way that, regardless of what the paperclip maximizer does, defecting gains one billion lives and loses two paperclips.
This is standard defense of defecting in a prisonner’s dilemma, but if it were valid then the dilemma wouldn’t be really a dilemma.
If you can assume that the maximizer uses the same decision algorithm as we do, we can also assume that it will come to the same conclusion. Given this, it is better to cooperate, since it will gain billion lives (and a paperclip). But we don’t know whether the paperclipper uses the same algorithm.
A.Crossman: Prase, Chris, I don’t understand. Eliezer’s example is set up in such a way that, regardless of what the paperclip maximizer does, defecting gains one billion lives and loses two paperclips. This is standard defense of defecting in a prisonner’s dilemma, but if it were valid then the dilemma wouldn’t be really a dilemma.
If you can assume that the maximizer uses the same decision algorithm as we do, we can also assume that it will come to the same conclusion. Given this, it is better to cooperate, since it will gain billion lives (and a paperclip). But we don’t know whether the paperclipper uses the same algorithm.