We can do adversarial training against other AIs, but ancestral humans didn’t have to contend with animals whose goal was to trick them into not reproducing by any means necessary
We did have to contend with memes that tried to hijack our minds to spread them horizontally (as opposed to vertically, by having more kids), but unfortunately (or fortunately) such “adversarial training” wasn’t powerful enough to instill a robust desire to maximize reproductive fitness. Our adversarial training for AI could also be very limited compared to the adversaries or natural distributional shifts the AI will face in the future.
Our fear of death is therefore much more robust than our desire to maximize reproductive fitness
My fear of death has been much reduced after learning about ideas like quantum immortality and simulation arguments, so it doesn’t seem that much more robust. Its apparent robustness in others looks like an accidental effect of most people not paying attention or being able to fully understand such ideas, which does not seem to have a relevant analogy for AI safety.
Quantum theory and simulation arguments both suggest that there are many copies of myself in the multiverse. From a first person subjective anticipation perspective, experiencing death as nothingness seems impossible so it seems like I should either anticipate my subjective experience continuing as one of the surviving copies, or the whole concept of subjective anticipation is confused. From a third person / God’s view, death can be thought of some of the copies being destroyed or a reduction in my “measure”, but I don’t seem to fear this, just as I didn’t jump in joy to learn about having a huge number of copies in the first place. The situation seems too abstract or remote or foreign to trigger my fear (or joy) response.
Does this meaningfully reduce the probability that you jump out of the way of a car or get screened for heart disease? The important thing isn’t whether you have an emotional fear response, but how the behavior pattern of avoiding generalizes.
We did have to contend with memes that tried to hijack our minds to spread them horizontally (as opposed to vertically, by having more kids), but unfortunately (or fortunately) such “adversarial training” wasn’t powerful enough to instill a robust desire to maximize reproductive fitness. Our adversarial training for AI could also be very limited compared to the adversaries or natural distributional shifts the AI will face in the future.
My fear of death has been much reduced after learning about ideas like quantum immortality and simulation arguments, so it doesn’t seem that much more robust. Its apparent robustness in others looks like an accidental effect of most people not paying attention or being able to fully understand such ideas, which does not seem to have a relevant analogy for AI safety.
Noted. Somewhat surprised you believe in quantum immortality, is there a particular reason?
Quantum theory and simulation arguments both suggest that there are many copies of myself in the multiverse. From a first person subjective anticipation perspective, experiencing death as nothingness seems impossible so it seems like I should either anticipate my subjective experience continuing as one of the surviving copies, or the whole concept of subjective anticipation is confused. From a third person / God’s view, death can be thought of some of the copies being destroyed or a reduction in my “measure”, but I don’t seem to fear this, just as I didn’t jump in joy to learn about having a huge number of copies in the first place. The situation seems too abstract or remote or foreign to trigger my fear (or joy) response.
Does this meaningfully reduce the probability that you jump out of the way of a car or get screened for heart disease? The important thing isn’t whether you have an emotional fear response, but how the behavior pattern of avoiding generalizes.