To be the same and to have the same goals are two distinct, but equally possible kinds of sameness.
Most humans seem to care much more about the former (survival) then the later (that their goals be sustained in the universe)
Citing Woody Allen: “I don’t want to achieve immortality through my work. I want to achieve it through not dying.”
We do have distinct reasons to think machine intelligences would like to preserve their goals, and that for them, perhaps identity would feel more entangled with goals, however those reasons are far from unequivocal.
I take this to be false.
To be the same and to have the same goals are two distinct, but equally possible kinds of sameness.
Most humans seem to care much more about the former (survival) then the later (that their goals be sustained in the universe)
Citing Woody Allen: “I don’t want to achieve immortality through my work. I want to achieve it through not dying.”
We do have distinct reasons to think machine intelligences would like to preserve their goals, and that for them, perhaps identity would feel more entangled with goals, however those reasons are far from unequivocal.