In a perfectly rational agent, no n-th order wants should exist.
Your problems with mountain dew might account for −1 util, you being awake for 2 utils, then you “want” to drink that stuff. Shut up and add.
The only source of multi-level desires I can see is an imperfect caching algorithm, which spews forth “Do not drink mountain dew” although the overall utility would be positive.
Your problems with mountain dew might account for −1 util, you being awake for 2 utils, then you “want” to drink that stuff. Shut up and add.
It still seems perfectly reasonable for a rational agent to not-want to not-want Mountain Dew here. If it were feasible to self-modify to become Moutain-Dew-indifferent at a cost less than 1 util, then the utility of drinking Mountain Dew & being indifferent to it would be greater than the utility of drinking it while continuing to not-want it.
Self-modifying not to feel discomfort from drinking Mountain Dew isn’t at all the same thing as self-modifying to want to drink Mountain Dew keeping discomfort constant. The latter isn’t something you want to do on pain of committing the Wirehead Fallacy. The former isn’t something that “want to want” language really applies to, as far as I can see.
I don’t see why not. Spicy foods inflict pain; wanting to develop a greater tolerance for cayenne pepper wouldn’t be all that weird. Mountain Dew inflicts pain; learning to put up with that would be instrumentally useful to me. I’d prefer that it just not inflict pain, but assuming that’s not an option, I’d be okay with just developing the ability to deal with it.
In case it hasn’t been mentioned, many stores that sell Mountain Dew also stock NoDoz tablets, which you can develop a technique for swallowing whole with water.
I was actually in the position of wanting to develop a tolerance for spicy foods when I was young. I would go to the pantry and treat myself by dosing myself with Tabasco sauce. But I don’t think of that as my “wanting to want KimChee.” I wanted to eat KimChee like my parents and I took steps to achieve that goal.
I don’t think there’s any need for meta-desires at all, except for one: a drive to resolve conflicting desires. And this is arguably also a 1st order desire. There are obvious reasons why we’d evolve such a drive. We can explain away things like wanting to want internal conflict as merely a desire to be someone like Timothy Levitch from “Waking Life.” It also makes sense that one can formulate paradoxical desires that are strongly resistant to resolution, or rationalize to oneself that they are conundrums. But I posit that these are merely ill-formed desires—that if you can’t reduce everything down to conflicting 1st-order desires, you haven’t delved deeply enough.
Agreed, what I said wasn’t literally true. If you get 2 utils from caffeine and −1 util from pain, and if despite this you don’t want to drink MD, then it’s rational to self-modify to want to drink MD. But the point I meant to make is that it’s not rational to self-modify to assign 0 utils to the same pain instead (because you don’t care about utils, you care about things using utils), which is what I (mis)interpreted conchis as saying.
I interpret “wanting to want” as encompassing the former, and don’t really see any reason to limit it in the way you suggest. But either way, we have no substantive disagreement.
Or the vertical hierarchy in our brains, and our inability to modify it. All four examples show “first order desires” created by lower level subsystems (taste perception, sexuality, motivations, etc.), and the higher level ones are trying to override it (consciousness, planning, models of the future world). But we aren’t designed rationally enough to be able to control those lower levels, although we know partially about what they “think”. (“We” = the conscious part writing LW comments...)
In a perfectly rational agent, no n-th order wants should exist.
Your problems with mountain dew might account for −1 util, you being awake for 2 utils, then you “want” to drink that stuff. Shut up and add.
The only source of multi-level desires I can see is an imperfect caching algorithm, which spews forth “Do not drink mountain dew” although the overall utility would be positive.
It still seems perfectly reasonable for a rational agent to not-want to not-want Mountain Dew here. If it were feasible to self-modify to become Moutain-Dew-indifferent at a cost less than 1 util, then the utility of drinking Mountain Dew & being indifferent to it would be greater than the utility of drinking it while continuing to not-want it.
Self-modifying not to feel discomfort from drinking Mountain Dew isn’t at all the same thing as self-modifying to want to drink Mountain Dew keeping discomfort constant. The latter isn’t something you want to do on pain of committing the Wirehead Fallacy. The former isn’t something that “want to want” language really applies to, as far as I can see.
I don’t see why not. Spicy foods inflict pain; wanting to develop a greater tolerance for cayenne pepper wouldn’t be all that weird. Mountain Dew inflicts pain; learning to put up with that would be instrumentally useful to me. I’d prefer that it just not inflict pain, but assuming that’s not an option, I’d be okay with just developing the ability to deal with it.
In case it hasn’t been mentioned, many stores that sell Mountain Dew also stock NoDoz tablets, which you can develop a technique for swallowing whole with water.
I was actually in the position of wanting to develop a tolerance for spicy foods when I was young. I would go to the pantry and treat myself by dosing myself with Tabasco sauce. But I don’t think of that as my “wanting to want KimChee.” I wanted to eat KimChee like my parents and I took steps to achieve that goal.
I don’t think there’s any need for meta-desires at all, except for one: a drive to resolve conflicting desires. And this is arguably also a 1st order desire. There are obvious reasons why we’d evolve such a drive. We can explain away things like wanting to want internal conflict as merely a desire to be someone like Timothy Levitch from “Waking Life.” It also makes sense that one can formulate paradoxical desires that are strongly resistant to resolution, or rationalize to oneself that they are conundrums. But I posit that these are merely ill-formed desires—that if you can’t reduce everything down to conflicting 1st-order desires, you haven’t delved deeply enough.
http://dannyman.toldme.com/2003/11/19/timothy-levitch-waking-life/
Agreed, what I said wasn’t literally true. If you get 2 utils from caffeine and −1 util from pain, and if despite this you don’t want to drink MD, then it’s rational to self-modify to want to drink MD. But the point I meant to make is that it’s not rational to self-modify to assign 0 utils to the same pain instead (because you don’t care about utils, you care about things using utils), which is what I (mis)interpreted conchis as saying.
I interpret “wanting to want” as encompassing the former, and don’t really see any reason to limit it in the way you suggest. But either way, we have no substantive disagreement.
Or the vertical hierarchy in our brains, and our inability to modify it. All four examples show “first order desires” created by lower level subsystems (taste perception, sexuality, motivations, etc.), and the higher level ones are trying to override it (consciousness, planning, models of the future world). But we aren’t designed rationally enough to be able to control those lower levels, although we know partially about what they “think”. (“We” = the conscious part writing LW comments...)