This conclusion is too strong, because there’s a clear distinction that we (or at least I) make intuitively that is incompatible with this reasoning.
Consider the following:
I don’t want to try sushi.
A friend convinces/bribes/coerces me to try sushi.
It turns out I really like sushi, and eat it all the time afterward.
I don’t want to try wireheading.
I am convinced/bribed/coerced to try wireheading.
I really like wireheading, and don’t want to stop doing it.
These sequences are superficially identical. Kaj’s construction of want suggests I could not have been mistaken about my desire for sushi. However, intuitively and in common language, it makes sense to say that I was mistaken about my desire for sushi. There is, however, something different about saying I was mistaken in not wanting to wirehead. It’s an issue of values.
Consider the ardent vegetarian who is coercively fed beef, and likes beef so much that he lacks the willpower to avoid eating it, even though it causes him tremendous psychic distress to do so. It seems reasonable to say he was correct in not wanting to eat beef, and have this judgement be entirely consistent with my being incorrect about not wanting to eat sushi. The issue is whether my action has a non-hedonic value. Eating sushi (for me) does not. Eating beef for him does. His hedonic values get in the way of his utilitarian values.
This dilemma actually integrates a number of rather complex problems. I’m hereby precommitting to making a top-level post about this before Friday. Let’s hope it works.
A possible solution to this: The person who does not want to try sushi thinks he will dislike it and say “Yuck!” He actually enjoys it. He is wrong in that he anticipated something different from what happened. A person who does not want to wirehead will anticipate enjoying it immensely, and this will be accurate. The first person’s decision to try to avoid sushi is based on a mistaken anticipation, but the second person’s decision to avoid wireheading takes into account a correct anticipation.
See my reply to zero_call below. Yes, in baseline humans and with current technology, it does make sense to use the expression “true desire”. As technology improves, however, you’ll need to define it more and more rigorously. Defining it by reference to your current values is one way.
This conclusion is too strong, because there’s a clear distinction that we (or at least I) make intuitively that is incompatible with this reasoning.
Consider the following:
I don’t want to try sushi. A friend convinces/bribes/coerces me to try sushi. It turns out I really like sushi, and eat it all the time afterward.
I don’t want to try wireheading. I am convinced/bribed/coerced to try wireheading. I really like wireheading, and don’t want to stop doing it.
These sequences are superficially identical. Kaj’s construction of want suggests I could not have been mistaken about my desire for sushi. However, intuitively and in common language, it makes sense to say that I was mistaken about my desire for sushi. There is, however, something different about saying I was mistaken in not wanting to wirehead. It’s an issue of values.
Consider the ardent vegetarian who is coercively fed beef, and likes beef so much that he lacks the willpower to avoid eating it, even though it causes him tremendous psychic distress to do so. It seems reasonable to say he was correct in not wanting to eat beef, and have this judgement be entirely consistent with my being incorrect about not wanting to eat sushi. The issue is whether my action has a non-hedonic value. Eating sushi (for me) does not. Eating beef for him does. His hedonic values get in the way of his utilitarian values.
This dilemma actually integrates a number of rather complex problems. I’m hereby precommitting to making a top-level post about this before Friday. Let’s hope it works.
A possible solution to this: The person who does not want to try sushi thinks he will dislike it and say “Yuck!” He actually enjoys it. He is wrong in that he anticipated something different from what happened. A person who does not want to wirehead will anticipate enjoying it immensely, and this will be accurate. The first person’s decision to try to avoid sushi is based on a mistaken anticipation, but the second person’s decision to avoid wireheading takes into account a correct anticipation.
No top level post? I has a sad.
And commitment devices work, if belatedly.
Yay!
See my reply to zero_call below. Yes, in baseline humans and with current technology, it does make sense to use the expression “true desire”. As technology improves, however, you’ll need to define it more and more rigorously. Defining it by reference to your current values is one way.