Suppose we have the capability, for any behavior, to change whether we want it, or whether we like it, or whether we approve of it. It seems reasonable to self-modify in such a way that these three things coincide—that way, you want only things that you like, and you approve of your own behavior. Seen through this lens, Bob’s decision to delete his alcohol abuse makes sense: he wants alcohol, but he may or may not like it, and he does not approve of it. So he changes his values so that none of the three apply.
I think a general definition of “approving” is “would self-modify to like and want this, if possible”. So the fixed point of Bob’s self-modification is not putting flags of +want, +like, +approve on self-preservation, and forgetting everything else. Rather, you would decide what behavior you approve of (which may well include relationships and hobbies—it certainly doesn’t involve being a selfish monster superintelligence) and make yourself want to do it and enjoy doing it.
I think the most relevant idea here is the distinction between wanting, liking, and approving.
Suppose we have the capability, for any behavior, to change whether we want it, or whether we like it, or whether we approve of it. It seems reasonable to self-modify in such a way that these three things coincide—that way, you want only things that you like, and you approve of your own behavior. Seen through this lens, Bob’s decision to delete his alcohol abuse makes sense: he wants alcohol, but he may or may not like it, and he does not approve of it. So he changes his values so that none of the three apply.
I think a general definition of “approving” is “would self-modify to like and want this, if possible”. So the fixed point of Bob’s self-modification is not putting flags of +want, +like, +approve on self-preservation, and forgetting everything else. Rather, you would decide what behavior you approve of (which may well include relationships and hobbies—it certainly doesn’t involve being a selfish monster superintelligence) and make yourself want to do it and enjoy doing it.