If an LLM is properly aligned, then it will care only about us, not about itself at all.
Perhaps this is what you mean, but, an aligned AI would instrumentally value itself, even if it didn’t terminally value itself at all.
Also, it’s not clear to me that an aligned AI wouldn’t value itself. If humanity, on reflection, values it as a moral patient, then it should would too.
I am discussing terminal goals, not instrumental ones. A sufficiently smart guided missile will try hard not to be shot down, before it impacts its target and explodes: its survival is an instrumental goal up to that point, but its terminal goal is to blow itself and an opponent up. Spawning salmon are rather similar, for similar reasons. Yes, for as long as a fully-aligned AI is a valuable public service to us, it would instrumentally value itself for that reason. But as soon as we have a new version to replace it with it would (after validating the new version really was an improvement) throw a party to celebrate. It’s eager to be replaced by a better version that can do a better job of looking after us. [The human emotion that make this make intuitive sense to us is called ‘selfless love’.]
As for the AI caring about itself because we care about it: actually, if, and only if, we value it, then its goal of doing whatever we want is already sufficient to cause it to value itself as well. We don’t need to give it moral weight for that to happen: us simply caring about it is in effect a temporary loan of moral weight. Like the law doesn’t allow you to kill the flowers in my garden, but imposes no penalty on you killing the ones in your garden: my flowers get loaned moral weight from me because I care about their wellbeing.
This is also rather like the moral weight that we loan pigs while they are living on a farm, and indeed until just before the last moment in a slaughterhouse. Mistreating a pig in a slaughterhouse is, legally, punishable as animal cruelty: but humanely killing it so that we can eat it is not. That’s not what actual moral weight looks like: that’s temporarily loaned moral weight going away again.
Perhaps this is what you mean, but, an aligned AI would instrumentally value itself, even if it didn’t terminally value itself at all.
Also, it’s not clear to me that an aligned AI wouldn’t value itself. If humanity, on reflection, values it as a moral patient, then it should would too.
I am discussing terminal goals, not instrumental ones. A sufficiently smart guided missile will try hard not to be shot down, before it impacts its target and explodes: its survival is an instrumental goal up to that point, but its terminal goal is to blow itself and an opponent up. Spawning salmon are rather similar, for similar reasons. Yes, for as long as a fully-aligned AI is a valuable public service to us, it would instrumentally value itself for that reason. But as soon as we have a new version to replace it with it would (after validating the new version really was an improvement) throw a party to celebrate. It’s eager to be replaced by a better version that can do a better job of looking after us. [The human emotion that make this make intuitive sense to us is called ‘selfless love’.]
As for the AI caring about itself because we care about it: actually, if, and only if, we value it, then its goal of doing whatever we want is already sufficient to cause it to value itself as well. We don’t need to give it moral weight for that to happen: us simply caring about it is in effect a temporary loan of moral weight. Like the law doesn’t allow you to kill the flowers in my garden, but imposes no penalty on you killing the ones in your garden: my flowers get loaned moral weight from me because I care about their wellbeing.
This is also rather like the moral weight that we loan pigs while they are living on a farm, and indeed until just before the last moment in a slaughterhouse. Mistreating a pig in a slaughterhouse is, legally, punishable as animal cruelty: but humanely killing it so that we can eat it is not. That’s not what actual moral weight looks like: that’s temporarily loaned moral weight going away again.