My claim is that it doesn’t seem complete to argue “we need a sense of identity to run long-run optimization problems well.” I run optimization programs without a sense of identity just fine- you tell them the objective function, you tell them the decision variables, you tell them the constraints, and then they process until they’ve got an answer. It doesn’t seem to me like you’re claiming the ‘sense of personal identity’ boils down to ‘the set of decision variables and the objective function,’ but I think that’s only as far as your argument goes.
Hmm, looks like I expressed myself badly, as several people seem to have this confusion. I wasn’t saying that long-term optimization problems in general would require a sense of identity, just that the specific optimization program that’s implemented in our current mental architecture seems to require it.
(Yes, a utilitarian could in principle decide that they want to minimize the amount of suffering in the world and then do a calculation about how to best achieve that which didn’t refer to a sense of identity at all… but they’ll have a hard time getting themselves to actually take action based on that calculation, unless they can somehow also motivate their more emotional predictive systems—which are based on a sense of personal identity—to also be interested in pursuing those goals.)
Hmm, looks like I expressed myself badly, as several people seem to have this confusion. I wasn’t saying that long-term optimization problems in general would require a sense of identity, just that the specific optimization program that’s implemented in our current mental architecture seems to require it.
(Yes, a utilitarian could in principle decide that they want to minimize the amount of suffering in the world and then do a calculation about how to best achieve that which didn’t refer to a sense of identity at all… but they’ll have a hard time getting themselves to actually take action based on that calculation, unless they can somehow also motivate their more emotional predictive systems—which are based on a sense of personal identity—to also be interested in pursuing those goals.)