In the universe I live in, there are both cooperators and defectors, but cooperators seem to predominate in random encounters. (If you leave yourself open to encounters in which others can choose to interact with you, defectors may find you an easy mark.)
In order to decide how to act with the paperclip maximizer, I have to figure out what kind of universe it is likely to inhabit. It’s possible that a random super intelligence from a random universe will have few opportunities to cooperate, but I think it’s more likely that there are far more SIs and universes in which cooperation is common.
But even though this is the direct answer to the question EY poses, I think it’s more important to point out that his is a better (though not simpler to explain as RH says) depiction of the intended dilemma. It takes much more thought to figure out what about the context would make cooperation reasonable. Viscerally, it’s nearly untenable.
In the universe I live in, there are both cooperators and defectors, but cooperators seem to predominate in random encounters. (If you leave yourself open to encounters in which others can choose to interact with you, defectors may find you an easy mark.)
In order to decide how to act with the paperclip maximizer, I have to figure out what kind of universe it is likely to inhabit. It’s possible that a random super intelligence from a random universe will have few opportunities to cooperate, but I think it’s more likely that there are far more SIs and universes in which cooperation is common.
But even though this is the direct answer to the question EY poses, I think it’s more important to point out that his is a better (though not simpler to explain as RH says) depiction of the intended dilemma. It takes much more thought to figure out what about the context would make cooperation reasonable. Viscerally, it’s nearly untenable.