I really like the core ideas of this post but some of the particulars are bothersome to me. For example, it confuses things IMO to talk about wireheading as though it can be modified to be whatever we want—wireheading is wireheading, and it has a rather clear, explicit meaning. (Although the degree of its strength would need to be qualified.)
Anyways, how do you really know what you want? That’s the really key question, which I don’t think you’ve really answered. It’s not just about redefining terms, IMO. There’s real substance to the idea that we have some innate, true sense of desires, yet whose identities elude us. To take the sushi example, the person who tries sushi and loves it had an innate desire, or interest, all along. It might not have been a “want”, but the fact that their preferences changed expresses something true about them. It wasn’t just a matter of definitions and perspectives and so on.
Maybe what you’re saying is that desires are somewhat irrelevant; they can be redefined, reupdated, or completely neglected, and they have little overall worth. So maybe the more interesting question is more straightforward: knowing we would be completely happy and fulfilled in a life of wireheading, should we do it?
wireheading is wireheading, and it has a rather clear, explicit meaning
We’ve assumed that it has a clear, explicit meaning, but I don’t think that’s so.
here’s real substance to the idea that we have some innate, true sense of desires, yet whose identities elude us.
In baseline humans and with current technology, yes, it does make sense to use the expression “true desire”. Not that particular desires would be any more “true” than others, but there may be some unrealized desires which, if fulfilled, would lead to the person becoming happier than if those desires weren’t fulfilled. As technology increases, that distinction becomes less meaningful, as we become capable of rebuilding our minds and transforming any desire to such a “true desire”.
If you wanted to keep the distinction even with improving technology, you’d define some class of alterations which are “acceptable” and some which aren’t. “True desires” would then be any wants that could be promoted to such a status using “acceptable” means. Wei Dai started compiling one possible list of such acceptable alterations.
I really like the core ideas of this post but some of the particulars are bothersome to me. For example, it confuses things IMO to talk about wireheading as though it can be modified to be whatever we want—wireheading is wireheading, and it has a rather clear, explicit meaning. (Although the degree of its strength would need to be qualified.)
Anyways, how do you really know what you want? That’s the really key question, which I don’t think you’ve really answered. It’s not just about redefining terms, IMO. There’s real substance to the idea that we have some innate, true sense of desires, yet whose identities elude us. To take the sushi example, the person who tries sushi and loves it had an innate desire, or interest, all along. It might not have been a “want”, but the fact that their preferences changed expresses something true about them. It wasn’t just a matter of definitions and perspectives and so on.
Maybe what you’re saying is that desires are somewhat irrelevant; they can be redefined, reupdated, or completely neglected, and they have little overall worth. So maybe the more interesting question is more straightforward: knowing we would be completely happy and fulfilled in a life of wireheading, should we do it?
We’ve assumed that it has a clear, explicit meaning, but I don’t think that’s so.
In baseline humans and with current technology, yes, it does make sense to use the expression “true desire”. Not that particular desires would be any more “true” than others, but there may be some unrealized desires which, if fulfilled, would lead to the person becoming happier than if those desires weren’t fulfilled. As technology increases, that distinction becomes less meaningful, as we become capable of rebuilding our minds and transforming any desire to such a “true desire”.
If you wanted to keep the distinction even with improving technology, you’d define some class of alterations which are “acceptable” and some which aren’t. “True desires” would then be any wants that could be promoted to such a status using “acceptable” means. Wei Dai started compiling one possible list of such acceptable alterations.