As I mentioned in the previous post, our values seem to be defined in terms of a world model where people exist as ontologically primitive entities ruled heuristically by (mostly intuitive understandings of) physics and psychology. In this kind of decision system, both identity-as-physical-continuity and identity-as-psychological-continuity make perfect sense as possible values, and it seems humans do “natively” have both values. A typical human being is both reluctant to step into a teleporter that works by destructive scanning, and unwilling to let their physical structure be continuously modified into a psychologically very different being.
If faced with the knowledge that physical continuity doesn’t exist in the real world at the level of fundamental physics, one might conclude that it’s crazy to continue to value it, and this is what Eliezer’s post argued. But if we apply this reasoning in a non-selective fashion, wouldn’t we also conclude that we should stop valuing things like “pain” and “happiness” which also do not seem to exist at the level of fundamental physics?
Physical continuity doesn’t really have any benefits that enter in at the level of our consciousness though. The fact that we don’t have pain on a particulate level doesn’t affect the sensation that you’ll feel if you grab a hot iron, whereas if the atoms of your hand are gradually replaced with other atoms, you’re not going to be gripped with the visceral sensation that it’s not your hand, and have a need to draw on some philosophical argument to try to disabuse yourself of that notion.
Physical continuity doesn’t really have any benefits that enter in at the level of our consciousness though. The fact that we don’t have pain on a particulate level doesn’t affect the sensation that you’ll feel if you grab a hot iron, whereas if the atoms of your hand are gradually replaced with other atoms, you’re not going to be gripped with the visceral sensation that it’s not your hand, and have a need to draw on some philosophical argument to try to disabuse yourself of that notion.