Incidentally, I should point out that in the economics and decision theory literature, “utility” is not a synonym for pleasure or some other psychological variable. It’s merely a mathematical representation of revealed preferences (preferences which may be motivated by an ultimate desire for pleasure, but that’s an additional substantive hypothesis). I tend to use “utility” in this sense, so just a terminological heads-up.
See here for some standard criticisms of hedonic (pleasure/pain based) utilitarianism.
Also see the discussions of wireheading on LW.
Incidentally, I should point out that in the economics and decision theory literature, “utility” is not a synonym for pleasure or some other psychological variable. It’s merely a mathematical representation of revealed preferences (preferences which may be motivated by an ultimate desire for pleasure, but that’s an additional substantive hypothesis). I tend to use “utility” in this sense, so just a terminological heads-up.