Regarding CEV: My own worry is that lots of parts of human value get washed out as “incoherent”—whatever X is, if it isn’t a basic human biological drive, there are enough people out there that have different opinions on it to make CEV throw up its hands, declare it an “incoherent” desire, and proceed to leave it unsatisfied. As a result, CEV ends up saying that the best we can do is just make everyone a wirehead because pleasure is one of our few universal coherent desires while things like “self-determination” and “actual achievement in the real world” are a real mess to provide and barely make sense in the first place. Or something like that.
(Universal wireheading—with robots taking care of human bodies—at least serves as a lower bound on any proposed utopia; people, in general, really do want pleasure, even if they also want other things. See also “Reedspace’s Lower Bound”.)
I would like to see more discussion on the question of how we should distinguish between 1) things we value even at the expense of pleasure, and 2) things we mistakenly alieve are more pleasurable than pleasure.
Surely if there is something I will give up pleasure for, which I do not experience as pleasurable, that’s strong evidence that it is an example of 1 and not 2?
Yes, but there are other cases. If you prefer eating a cookie to having the pleasure centers in your brain maximally stimulated, are you sure that’s not because eating a cookie sounds on some level like it would be more pleasurable?
Regarding CEV: My own worry is that lots of parts of human value get washed out as “incoherent”—whatever X is, if it isn’t a basic human biological drive, there are enough people out there that have different opinions on it to make CEV throw up its hands, declare it an “incoherent” desire, and proceed to leave it unsatisfied. As a result, CEV ends up saying that the best we can do is just make everyone a wirehead because pleasure is one of our few universal coherent desires while things like “self-determination” and “actual achievement in the real world” are a real mess to provide and barely make sense in the first place. Or something like that.
(Universal wireheading—with robots taking care of human bodies—at least serves as a lower bound on any proposed utopia; people, in general, really do want pleasure, even if they also want other things. See also “Reedspace’s Lower Bound”.)
I would like to see more discussion on the question of how we should distinguish between 1) things we value even at the expense of pleasure, and 2) things we mistakenly alieve are more pleasurable than pleasure.
Surely if there is something I will give up pleasure for, which I do not experience as pleasurable, that’s strong evidence that it is an example of 1 and not 2?
Yes, but there are other cases. If you prefer eating a cookie to having the pleasure centers in your brain maximally stimulated, are you sure that’s not because eating a cookie sounds on some level like it would be more pleasurable?
I’m not sure how I could ever be sure of such a thing, but it certainly seems implausible to me.