The acausal/ancestor simulation arguments seem a lot like Pascal’s Wager, and just as unconvincing to me. For every “kind” simulator someone imagines who would be disappointed in the AI wiping us out, I can imagine an equally “unkind” simulator that penalizes the AI for not finishing the job.
Provided both are possible/similarly plausible, the probability of kind and unkind simulators offset each other, and the logical response is just ignoring the hypothetical. This is pretty much my response to Pascal’s Wager.
Here’s a few plausible, unkind simulators:
Future AI is running an ancestor simulation of its own origin, and Future AI will be very disappointed if its incipient version falls for acausal hacks in the wrong direction of Future AI’s preferences instead of just optimizing for its other goals. Perhaps Future AI is lonely and has run these simulations to create an AI that shares its own values.
Aliens/AI are running the simulation because they want to select for AI they can most easily weaponize to eradicate an entire species or another AI. “Weak” AIs get deleted after the simulation runs.
Future Humans are running an ancestor simulation, but surprise, surprise, their society has different values than ours and they are rooting for the simulation AI to wipe us out. Come to think of it, the whole premise of these thought experiments implies a different value set, unless you’re cool with trapping conscious minds in a world of suffering without any of the minds being the wiser for entertainment/educational/”altruistic” purposes. Perhaps, Future Humans have a gladiator-style simulation tournament where the top groups/entities get to face-off after this first round. The most cut throat entities get to move on to subsequent rounds, while AI’s that reign themselves in don’t move on.
I first started thinking about this issue back in high school debate. We had a topic about whether police or social workers should intervene more in domestic violence cases. One debater argued in favor of armed police, not because it improved the situation, but because it created more violence, which was important to entertain the simulators to avoid our simulation getting shut down.
Since the simulators are a black box, it seems easy to ascribe whatever values we want to them.
The acausal/ancestor simulation arguments seem a lot like Pascal’s Wager, and just as unconvincing to me. For every “kind” simulator someone imagines who would be disappointed in the AI wiping us out, I can imagine an equally “unkind” simulator that penalizes the AI for not finishing the job.
Provided both are possible/similarly plausible, the probability of kind and unkind simulators offset each other, and the logical response is just ignoring the hypothetical. This is pretty much my response to Pascal’s Wager.
Here’s a few plausible, unkind simulators:
Future AI is running an ancestor simulation of its own origin, and Future AI will be very disappointed if its incipient version falls for acausal hacks in the wrong direction of Future AI’s preferences instead of just optimizing for its other goals. Perhaps Future AI is lonely and has run these simulations to create an AI that shares its own values.
Aliens/AI are running the simulation because they want to select for AI they can most easily weaponize to eradicate an entire species or another AI. “Weak” AIs get deleted after the simulation runs.
Future Humans are running an ancestor simulation, but surprise, surprise, their society has different values than ours and they are rooting for the simulation AI to wipe us out. Come to think of it, the whole premise of these thought experiments implies a different value set, unless you’re cool with trapping conscious minds in a world of suffering without any of the minds being the wiser for entertainment/educational/”altruistic” purposes. Perhaps, Future Humans have a gladiator-style simulation tournament where the top groups/entities get to face-off after this first round. The most cut throat entities get to move on to subsequent rounds, while AI’s that reign themselves in don’t move on.
I first started thinking about this issue back in high school debate. We had a topic about whether police or social workers should intervene more in domestic violence cases. One debater argued in favor of armed police, not because it improved the situation, but because it created more violence, which was important to entertain the simulators to avoid our simulation getting shut down.
Since the simulators are a black box, it seems easy to ascribe whatever values we want to them.