Putting “cultural change” and “an alien species comes along and murders us into extinction” into the same bucket seems like a mistake to me
Value drift encompasses a lot more than cultural change. If you think humans messing up on alignment could mean something as dramatic as “an alien species comes along and murders us”, surely you should think that the future could continue to include even more, similarly dramatic shifts. Why would we assume that once we solve value alignment for the first generation of AIs, values would then be locked in perfectly forever for all subsequent generations?
Alice, who is 10 years old: “Ah, I am about to be hit by a car, I should make sure this doesn’t happen else I’ll never get to live the rest of my life!”
Bob: “But have you considered that in the future you might get hit by another car? Or get cancer? Or choke on an olive? Current human life expectancy is around 80 years which means later on you’ll very likely die from something else. So I wouldn’t say your life is at stake right now.”
I don’t understand how this new analogy is supposed to apply to the argument, but if I wanted to modify the analogy to get my point across, I’d make Alice 90 years old. Then, I’d point out that, at such an age, getting hit by a car and dying painlessly genuinely isn’t extremely bad, since the alternative is to face death within the next several years with high probability anyway.
Value drift encompasses a lot more than cultural change. If you think humans messing up on alignment could mean something as dramatic as “an alien species comes along and murders us”, surely you should think that the future could continue to include even more, similarly dramatic shifts. Why would we assume that once we solve value alignment for the first generation of AIs, values would then be locked in perfectly forever for all subsequent generations?
I don’t understand how this new analogy is supposed to apply to the argument, but if I wanted to modify the analogy to get my point across, I’d make Alice 90 years old. Then, I’d point out that, at such an age, getting hit by a car and dying painlessly genuinely isn’t extremely bad, since the alternative is to face death within the next several years with high probability anyway.