I’d rather see ‘consequentialist’ supplemented or replaced by specific questions that get at substantive ethical or meta-ethical disputes in EA and philosophy. ‘Utilitarian’ and ‘deontologist’ mean lots of different things to different people, and on their strictest definitions they don’t entail a lot of their most interesting or widely cited ideas. Perhaps have an exploratory question one year asking non-utilitarians to write in their main objection to utilitarianism, then convert that into a series of questions the following year.
This was something I suggested to Tom because I’d be interested too. But ultimately we thought that only a small group of EAs would really have substantive ethical opinions and we thought to trim things for survey length. We added a box asking for clarifications at the end of the survey to provide more of this outlet.
One of the main objections to utilitarianism, it seems to me, is skepticism about the possibility (or even coherence of the notion) of aggregating utility across individuals. That’s one of my main objections, at any rate.
Skepticism about the applicability of the VNM theorem to human preferences is another issue, though that one might be less widespread.
Edit: The SEP describes classic utilitarianism as actual, direct, evaluative, hedonistic, maximizing, aggregative (specifically, total), universal, equal-consideration, agent-neutral consequentialism. I have definite issues with the “actual”, “direct”, “hedonistic”, “aggregative”, “total”, and “equal-consideration” parts of that. (Though I expect that my issues with “actual” will be shared by a significant portion of those who consider themselves utilitarians here, and my issues with “hedonistic” and “direct” may be as well. That leaves “aggregative”+”total”, and “equal-consideration”, as the two aspects most likely to be sources of philosophical conflict.)
Those sound like objections to preference utilitarianism but not hedonistic utilitarianism. Although it’s not technically possible yet, measuring the intensity of the positive and negative components of an experience sounds something that ought to be at least possible in principle. And the applicability of the VNM theorem to human preferences becomes irrelevant if you’re not interested in preferences in the first place.
Yes, true enough[1]; I did not properly separate those objections in my comment. To elaborate:
I object to hedonistic utilitarianism on the grounds that it clearly and grossly fails to capture my moral intuitions or those of anyone else whom I consider not to be evading the question. A full takedown of the “hedonistic” part of “hedonistic utilitarianism” is basically (at least) all of Eliezer’s posts about the complexity of value and so forth, and I won’t rehash it here.
To be honest, hedonistic utilitarianism seems to me to be so obviously wrong that I’m not even all that interested in having this sort of moral philosophy debate with an effective altruist (or anyone else) who holds such a view. I mean, to start with, my hypothetical interlocutor would have to rebut all the objections raised to hedonistic utilitarianism over the centuries since it’s been articulated, including, but not limited to, the aforementioned Lesswrong material.
I object to preference utilitarianism because of the “aggregation of utility” and “possibility of constructing a utility function” issues[2]. I think this is the more interesting objection.
[1] I’m not sure “intensity of the positive and negative components of an experience” is a coherent notion. There may not be a single quantity like that to measure. And even if we can measure something which we think qualifies for the title, it may be measurable only in some more-or-less absolute terms, while leaving open the question of how this hypothetical measured quantity matches up with anything like “utility to this particular experiencer”. But, for the sake of the argument, I’m willing to grant that such a quantity can indeed be usefully measured, because this is certainly not my true rejection.
[2] These are my objections to the “preference” component of preference utilitarianism; my objection to classical utilitarianism also includes objections to other components, which I have enumerated in the grandparent.
1) Even if hedonistic utilitarianism would ultimately be wrong as a full description of what a person values, “maximize pleasure while minimizing suffering” can still be a useful heuristic to follow. Yes, following that heuristic to its logical conclusion would mean forcibly rewiring everyone’s brains, but that doesn’t need to be a problem for as long as forcibly rewiring people’s brains isn’t a realistic option. HU may still be the best approximation of a person’s values in the context of today’s world, even if it wasn’t the best description overall.
2) The arguments on complexity of value and so on establish that the average person’s values aren’t correctly described by HU. This still leaves open the possibility of someone only approving of those of their behaviors that serve to promote HU, so there may definitely be individual people who accept HU, due to not sharing the moral intuitions which motivate the objections to it.
On 1): I am skeptical of replies to the effect that “yes, well, X might not be quite right, but it’s a useful heuristic, therefore I will go on acting as if X is right”. For one thing, a person who makes such a reply usually goes right back to saying “X is right!” (sans qualifiers) as soon as the current conversation ends. Let’s get clear on what we actually believe, I generally think; once we’ve firmly established that, we can look for maximally effective implementations.
For another thing, HU may be the best approximation etc. etc., but that’s a claim that at least should be made explicitly, such that it can be examined and argued for; a claim of this importance shouldn’t come up only in such tangential discussion branches.
For a third thing, what happens when forcibly rewiring people’s brains becomes a realistic option?
On 2): I think there’s two issues here. There could indeed be people who accept HU because that’s what correctly describes their moral intuitions. (Though I should certainly hope they do not think it proper to impose that moral philosophy on me, or on anyone else who doesn’t subscribe to HU!)
“Only approving of those behaviors that serve to promote HU” is, I think, a separate thing. Or at least, I’d need to see the concept expanded a bit more before I could judge. What does this hypothetical person believe? What moral intuitions do they have? What exactly does it mean to “promote” hedonistic utilitarianism?
There could indeed be people who accept HU because that’s what correctly describes their moral intuitions. (Though I should certainly hope they do not think it proper to impose that moral philosophy on me, or on anyone else who doesn’t subscribe to HU!)
Why would this be improper? Don’t that it doesn’t follow from any meta-ethical position.
If you say “all that matters is pain and pleasure”, and I say “no! I care about other things!”, and you’re like “nope, not listening. PAIN AND PLEASURE ARE THE ONLY THINGS”, and then proceed to enact policies which minimize pain and maximize pleasure, without regard for any of the other things that I care about, and all the while I’m telling you that no, I care about these other things! Stop ignoring them! Other things matter to me! but you’re not listening because you’ve decided that only pain and pleasure can possibly matter to anyone, despite my protestations otherwise...
… well, I hope you can see how that would bother me.
It’s not just a matter of us caring about different things. If it were only that, we could acknowledge the fact, and proceed to some sort of compromise. Hedonistic utilitiarians, however, do not acknowledge that it’s possible, or that it’s valid, to care about things that are not pain or pleasure. All these people who claim to care about all sorts of other things must be misguided! Clearly.
Hedonistic utilitiarians, however, do not acknowledge that it’s possible, or that it’s valid, to care about things that are not pain or pleasure.
They may think it’s incorrect if they’re realists, or cognitivists of some other form. But this has nothing to do with their being HUs, only with their being cognitivists.
[Description of situation] … well, I hope you can see how that would bother me.
Here are 3 non-exhaustive ways in which the situation you described could be bothersome:
(i) If your first order ethical theory (as opposed to your meta-ethics), perhaps combined with very plausible facts about human nature, requires otherwise. For instance if it speaks in favour of toleration or liberty here.
(ii) If you’re a cognitivist of the sort who thinks she could be wrong, it could increase your credence that you’re wrong.
(iii) If you’d at least on reflection give weight to the evident distress SaidAchmiz feels in this scenario, as most HUs would.
Hedonistic utilitiarians, however, do not acknowledge that it’s possible, or that it’s valid, to care about things that are not pain or pleasure.
They may think it’s incorrect if they’re realists, or cognitivists of some other form. But this has nothing to do with their being HUs, only with their being cognitivists.
No, I don’t think this is right. I think you (and Kaj_Sotala) are confusing these two questions:
Is it correct to hold an ethical view that is something other than hedonistic utilitarianism?
Does it make any sense to intrinsically value anything other than pleasure, or intrinsically disvalue things other than pain?
#1 is a meta-ethical question; moral realism or cognitivism may lead you to answer “no”, if you’re a hedonistic utilitarian. #2 is an ethical question; it’s about the content of hedonistic utilitarianism.
If I intrinsically care about, say, freedom, that’s not an ethical claim. It’s just a preference. “Humans may have preferences about things other than pain/pleasure, and those preferences are morally important” is an ethical claim which I might formulate, about that preference that I have.
Hedonistic utilitarianism tells me that my aforementioned preference is incoherent or mistaken, and that in fact I do not have any preferences (or any preferences that are morally important or worth caring about) other than preferences about pleasure/pain.
Moral realism (which, as blacktrance correctly notes, is implied by any utilitarianism) may lead a hedonistic utilitarian to say that my aforementioned ethical claim is incorrect.
As for your scenarios, I’m not sure what you meant by listing them. My point was that my scenario, which describes a situation involving a hypothetical me, Said Achmiz, would be bothersome to me, Said Achmiz. Is it really not clear why it would be?
If I intrinsically care about, say, freedom, that’s not an ethical claim. It’s just a preference. [...]
Hedonistic utilitarianism tells me that my aforementioned preference is incoherent or mistaken, and that in fact I do not have any preferences (or any preferences that are morally important or worth caring about) other than preferences about pleasure/pain.
Ethical subjectivism (which I subscribe to) would say that “ethical claims” are just a specific subset of our preferences; indeed, I’m rather skeptical of the notion of there being a distinction between ethical claims and preferences in the first place. But HU wouldn’t necessarily say that someone’s preference for something else than pleasure or pain would be mistaken—if it’s interpreted within a subjectivist framework, HU is just a description of preferences that are different. See my response to blacktrance.
But HU wouldn’t necessarily say that someone’s preference for something else than pleasure or pain would be mistaken—if it’s interpreted within a subjectivist framework, HU is just a description of preferences that are different.
I really don’t think that this is correct. If this were true, first of all, hedonistic utilitarianism would simply reduce to preference utilitarianism. In actual fact, neither view is merely about one’s own terminal values.
If someone, personally, cares only about pain and pleasure, but acknowledges that other people may have other things as terminal values, and thinks that The Good lies in satisfying everyone’s preferences maximally — which, for themselves, means maximizing pleasure and minimizing pain, and for other people may mean other things — then that person is not a hedonistic utilitarian. They are a preference utilitarian. Referring to them as an HU is simply not correct, because that’s not how the term is used in the philosophical literature.
On the other hand, if someone cares only about pain and pleasure — both theirs and other peoples’ — and would prefer that everyone’s pleasure be maximized and everyone’s pain be minimized; but this person is not a moral realist, and has no opinion on what constitutes The Good or thinks there’s no fact of the matter about whether an act is right or wrong; well, then this person is not a utilitarian at all. Again, describing this person as a hedonistic or any other kind of utilitarian completely fails to match up with how the term is used in the philosophical literature.
As for ethical subjectivism — uh, I don’t think that’s an actual thing. I’d not heard of anything by that name until today. I don’t like going by wikipedia’s definitions of philosophical principles, so I tried tracking it down to a source, such as perhaps a major philosopher espousing the view or at least describing it coherently. No such luck. Take a look at that list of references on its wikipedia page; two are to a single book (written in 1959 by some guy I’ve never heard of — have you? — and the shortness whose wikipedia page suggests that he wasn’t anyone interesting), and one is to a barely-related page that mentions the thing once, in passing, by a different name. I’m not convinced. As best I can tell, it’s a label that some modern-day historians of philosophy have used to describe… a not-quite-consistent family of views. (Divine command theory, for one.)
But let’s attempt to take it at face value. You say:
Someone could be an ethical subjectivist and say that utilitarianism is the theory that best describes their particular attitudes, or at least that subset of their attitudes that they endorse.
Very well. Are their attitudes correct, do they think? If they say there’s no fact of the matter about that, then they’re not a utilitiarian. “Utilitiarianism” is a quite established term in the literature. You can’t just apply it to any old thing.
Of course, this is Lesswrong; we don’t argue about definitions; we’re interested in what people actually think. However in this case I think getting our terms straight is important, for two reasons:
When most people say they’re utilitarians, they mean it in the usual sense, I think. So to understand what’s going on in these discussions, and in the heads of the people we’re talking to, we need to know what is the usual sense.
If you hold some view which is not one of the usual views with commonly-known terms, you shouldn’t call it by one of the commonly-known terms, because then I won’t have any idea what you’re talking about and we’ll keep getting into comment threads like this one.
On the other hand, if someone cares only about pain and pleasure — both theirs and other peoples’ — and would prefer that everyone’s pleasure be maximized and everyone’s pain be minimized; but this person is not a moral realist, and has no opinion on what constitutes The Good or thinks there’s no fact of the matter about whether an act is right or wrong; well, then this person is not a utilitarian at all. Again, describing this person as a hedonistic or any other kind of utilitarian completely fails to match up with how the term is used in the philosophical literature.
You may be right to say that my use of “utilitarian” is different from how it’s conventionally used in the literature
… though, I just looked at the SEP entry on Consequentialism, and I note that aside for the title of one book in the bibliography, nowhere in the article is the word “realism” even mentioned. Nor does there seem to be an entry in the list of claims making up classic utilitarianism that would seem to require moral realism. I guess you could kind of interpret one of these three conditions as requiring moral realism:
Universal Consequentialism = moral rightness depends on the consequences for all people or sentient beings (as opposed to only the individual agent, members of the individual’s society, present people, or any other limited group).
Equal Consideration = in determining moral rightness, benefits to one person matter just as much as similar benefits to any other person (= all who count count equally).
Agent-neutrality = whether some consequences are better than others does not depend on whether the consequences are evaluated from the perspective of the agent (as opposed to an observer).
… but it doesn’t seem obvious to me why someone who was both an ethical subjectivist couldn’t say that “I’m a classical utiliarian, in that (among other things) the best description of my ethical system is that I think that the goodness of an action should be determined based on how it affects all sentient beings, that benefits to one person matter just as much as similar benefits to others, and that the perspective of the people evaluating the consequences doesn’t matter. Though of course others could have ethical systems that were not well described by these items, and that wouldn’t make them wrong”.
Or maybe the important part in your comment was the part ”...but this person is not a moral realist, and has no opinion on what constitutes The Good”? But a subjectivist doesn’t say that he has no opinion on what constitutes The Good: he definitely has an opinion, and there may clearly be a right and wrong answer with regard to the kind of actions that are implied by his personal moral system; it’s just that the thing that constitutes The Good will be different for people with different moral systems.
Consequenialism supplies a realistic ontology, since it’s goods are facts about the real world, and utilitarian supplies an objective epistemology, since different utilitarians of the same stripe can converge. That adds up to some of the ingredients of realism, but not all of them. What is specifically lacking is an justification of comsequentialist ends as being objectively good, and not just subjectively desirable.
Consequenialism supplies a realistic ontology, since it’s goods are facts about the real world,
For this to make it realist, the fact that the truth of those facts has value would also have to be mind-independent. Even subjectivists typically value facts about the external world (e.g. their pleasure).
(I like this quote from that article, btw: “So many debates in philosophy revolve around the issue of objectivity versus subjectivity that one may be forgiven for assuming that someone somewhere understands this distinction.”)
You may be right to say that my use of “utilitarian” is different from how it’s conventionally used in the literature; I’m pretty unfamiliar with the actual ethical literature. But if we have people who have the attitude of “I want to take the kinds of actions that maximally increase pleasure and maximally reduce suffering and I’m a moral realist” and people who have the attitude of “I want to take the kinds of actions that maximally increase pleasure and maximally reduce suffering and I’m a moral non-realist”, then it feels a little odd to have different terms for them, given that they probably have more in common with each other (with regard to the actions that they take and the views that they hold) than e.g. two people who are both moral realists but differ on consequentialism vs. deontology.
At least in a context where we are trying to categorize people into different camps based on what they think we should actually do, it would seem to make sense if we just called both the moral realist and moral non-realist “utilitarians”, if they both fit the description of a utilitarian otherwise.
Hedonistic utilitiarians, however, do not acknowledge that it’s possible, or that it’s valid, to care about things that are not pain or pleasure. All these people who claim to care about all sorts of other things must be misguided!
I don’t think that hedonistic utilitarianism necessarily implies moral realism. Some HUs will certainly tell you that the people who morally disagree with them are misguided, but I don’t see why the proportion of HUs who think so (vs. the proportion of HUs who think that you are simply caring about different things) would need to be any different than it would be among the adherents of any other ethical position.
Maybe you meant your comment to refer specifically to the kinds of HUs who would impose their position on you, but even then the moral realism doesn’t follow. You can want to impose your values on others despite thinking that values are just questions of opinion. For instance, there are things that I consider basic human rights and I want to impose the requirement to respect them on every member of every society, even though there are people who would disagree with that requirement. I don’t think that the people who disagree are misguided in any sense, I just think that they value different things.
I agree with blacktrance’s reply to you, and also see my reply to tog in a different subthread for some commentary. However, I’m sufficiently unsure of what you’re saying to be certain that your comment is fully answered by either of those things. For example:
HUs who think that you are simply caring about different things
If you [the hypothetical you] think that it’s possible to care (intrinsically, i.e. terminally) about things other than pain and pleasure, then I’m not quite sure how you can remain a hedonistic utilitarian. You’d have to say something like: “Yes, many people intrinsically value all sorts of things, but those preferences are morally irrelevant, and it is ok to frustrate those preferences as much as necessary, in order to minimize pain and maximize pleasure.” You would, in other words, have to endorse a world where all the things that people value are mercilessly destroyed, and the things they most abhor and despise come to pass, if only this world had the most pleasure and least pain.
Now, granted, people sometimes endorse the strangest things, and I wouldn’t even be surprised to find someone on Lesswrong who held such a view, but then again I never claimed otherwise. What I said was that I should hope those people do not impose such a worldview on me.
If I’ve misinterpreted your comment and thereby failed to address your points, apologies; please clarify.
If you [the hypothetical you] think that it’s possible to care (intrinsically, i.e. terminally) about things other than pain and pleasure, then I’m not quite sure how you can remain a hedonistic utilitarian.
Well, if you’re really curious about how one could be a hedonistic utilitarian while also thinking that it’s possible to care intrinsically about things other than pain and pleasure, one could think something like:
“So there’s this confusing concept called ‘preferences’ that seems to be a general term for all kinds of things that affect our behavior, or mental states, or both. Probably not all the things that affect our behavior are morally important: for instance, a reflex action is a thing in a person’s nervous system that causes them to act in a certain way in certain situations, so you could kind of call that a preference to act in such a way in such a situation, but it still doesn’t seem like a morally important one.
“So what does make a preference morally important? If we define a preference as ‘an internal disposition that affects the choices that you make’, it seems like there would exist two kinds of preferences. First there are the ones that just cause a person to do things, but which don’t necessarily cause any feelings of pleasure or pain. Reflexes and automated habits, for instance. These don’t feel like they’d be worth moral consideration any more than the automatic decisions made by a computer program would.
“But then there’s the second category of preferences, ones that cause pleasure when they are satisfied, suffering when they are frustrated, or both. It feel like pleasure is a good thing and suffering is a bad thing, so that makes it good to satisfy the kinds of preferences that are produce pleasure when satisfied, as well as bad to frustrate the kinds of preferences that cause suffering when frustrated. Aha! Now I seem to have found a reasonable guideline for the kinds of preferences that I should care about. And of course this goes for higher-order preferences as well: if someone cares about X, then trying to change that preference would be a bad thing if they had a preference to continue caring about X, such that they would feel bad if someone tried to change their caring about X.
“And of course people can have various intrinsic preferences for things, which can mean that they do things even though that doesn’t produce them any suffering or pleasure. Or it can mean that doing something gives them pleasure or lets them avoid suffering by itself, even when doing that something doesn’t lead to any other consequence. The first kind of intrinsic preference I already concluded was morally irrelevant; the second kind is worth respecting, again because violating it would cause suffering, or reduce pleasure, or both. And I get tired of saying something clumsy like ‘increasing pleasure and decreasing suffering’ all the time, so let’s just call that ‘increasing well-being’ for short.
“Now unfortunately people have lots of different intrinsic preferences, and they often conflict. We can’t satisfy them all, as nice as it would be, so I have to choose my side. Since I chose my favored preferences on the basis that pleasure is good and suffering is bad, it would make sense to side with the preferences that, in the long term, produce the greatest amount of well-being in the world. For instance, some people may want the freedom to lie and cheat and murder, whereas other people want to have a peaceful and well-organized society. I think the preferences for living in peace will lead to greater well-being in the long term, so I will side with them, even if that means that the preferences of the sociopaths and murderers will be frustrated.
“Now there’s also this kind of inconvenient issue that if we rewire people’s brains so that they’ll always experience the maximal amount of pleasure, then that will produce more well-being in the long run, even if those people don’t currently want to have their brains rewired. I previously concluded that I should side with kinds of preferences that produce the greatest amount of well-being in the world, and the preference of ‘let’s rewire everyone’s brains’ does seem to produce by far the greatest amount of well-being in the world. So I should side with that preference, even though it goes against the intrinsic preferences of a lot of other people, but so did the decision to impose a lawful and peaceful society on the sociopaths and murderers, so that’s okay by me.
“Of course, other people may disagree, since they care about different things than pain and pleasure. And they’re not any more or less right—they just have different criteria for what counts as a moral action. But if it’s either them imposing their worldview on me, or me imposing my worldview on them, well, I’d rather have it be me imposing mine on them.”
I wouldn’t even be surprised to find someone on Lesswrong who held such a view, but then again I never claimed otherwise. What I said was that I should hope those people do not impose such a worldview on me.
Right, I wasn’t objecting to your statement of not wanting to have such a worldview imposed on you. I was only objecting to the statement that hedonistic utilitarians would necessarily have to think that others were misguided in some sense.
Any form of utilitarianism implies moral realism, as utilitarianism is a normative ethical theory and normative ethical theories presuppose moral realism.
I feel that this discussion is rapidly descending into a debate over definitions, but as a counter-example, take ethical subjectivism, which is a form of moral non-realism and which Wikipedia defines as claiming that:
Ethical sentences express propositions.
Some such propositions are true.
Those propositions are about the attitudes of people.
Someone could be an ethical subjectivist and say that utilitarianism is the theory that best describes their particular attitudes, or at least that subset of their attitudes that they endorse.
Someone could be an ethical subjectivist and want to maximize world utility, but such a person would not be a utilitarian, because utilitarianism holds that other people should maximize world utility. If you merely say “I want to maximize world utility and others to do the same”, that is not utilitarianism—a utilitarian would say that you ought to maximize world utility, even if you don’t want to, and it’s not a matter of attitudes. Yes, this is arguing over definitions to some extent, but it’s important because I often see this kind of confusion about utilitarianism on LW.
Could you provide a reference for that? At least the SEP entry on the topic doesn’t clearly state this. I’m also unsure of what difference this makes in practice—I guess we could come up with a new word for all the people who are both moral antirealist and utilitarian-aside-for-being-moral-antirealists, but I’m not sure if the difference in their behavior and beliefs is large enough for that to be worth it.
The SEP entry for consequentialism says it “is the view that normative properties depend only on consequences”, implying a belief in normative properties, which means moral realism.
If you want to describe people’s actions, a utilitarian and a world-utility-maximizing non-realist would act similarly, but there would be differences in attitude: a utilitarian would say and feel like he is doing the morally right thing and those who disagree with him are in error, whereas the non-realist would merely feel like he is doing what he wants and that there is nothing special about wanting to maximize world utility—to him, it’s just another preference, like collecting stamps or eating ice cream.
A non-consequentialist could be a moral realist as well, such as if they were a deontologist, so it’s not a good measurement.
Also, consequentialism and moral realism aren’t always well-defined terms.
Edit: That survey’s results are strange. Twenty people answered that they’re moral realists but non-cognitivists, though moral realism is necessarily cognitivist.
Let’s get clear on what we actually believe, I generally think; once we’ve firmly established that, we can look for maximally effective implementations.
For another thing, HU may be the best approximation etc. etc., but that’s a claim that at least should be made explicitly
I agree that it would often be good to be clearer about these points.
For a third thing, what happens when forcibly rewiring people’s brains becomes a realistic option?
At that point the people who consider themselves hedonistic utilitarians might come up with a theory that says that forcible wireheading is wrong and switch to calling themselves supporters of that theory. Or they could go on calling themselves HUs despite not forcibly wireheading anyone, in the same way that many people call themselves utilitarians today despite not actually giving most of their income away. Or some of them could decide to start working towards efforts to forcibly wirehead everyone, in which case they’d become the kinds of people described by my reply 2).
“Only approving of those behaviors that serve to promote HU” is, I think, a separate thing. Or at least, I’d need to see the concept expanded a bit more before I could judge.
By this, I meant to say “only approve of whatever course of action HU says is the best one”.
In that case, I’m unsure of what kind of an answer you were expecting (unless the “what then” was meant as a rhetorical question, but even then I’m slightly unsure of what point it was making).
Yes, the “what then” was rhetorical. If I had to express my point non-rhetorically, it’d be something like this:
If you take a position which gives ethically correct results only until such time as some (reasonably plausible) scenario comes to pass, then maybe your position isn’t ethical in the first place. “This ethical framework gives nonsensical or monstrous results in edge cases [of varying degrees of edge-ness]” is, after all, a common and quite justified criticism of ethical frameworks.
If you take a position which gives ethically correct results only until such time as some (reasonably plausible) scenario comes to pass, then maybe your position isn’t ethical in the first place. “This ethical framework gives nonsensical or monstrous results in edge cases [of varying degrees of edge-ness]” is, after all, a common and quite justified criticism of ethical frameworks.
It is a point against the framework, certainly. But so far nobody has developed an ethical framework that would have no problems at all, so at the moment we can only choose the framework that’s the least bad.
(Assuming that we wish to choose one in the first place, of course—I do think that there is merit in just accepting that they’re all flawed and then not choosing to endorse any single one.)
(Assuming that we wish to choose one in the first place, of course—I do think that there is merit in just accepting that they’re all flawed and then not choosing to endorse any single one.)
Well, that’s been my policy so far, certainly. Some are worse than others, though. “This ethical framework breaks in catastrophic, horrifying fashion, creating an instant dystopia, as soon as we can rewire people’s brains” is pretty darn bad.
Wireheading. The term is not a metaphor, and it’s not a hypothetical. You can literally stick a wire into someone’s pleasure centers and activate them, using only non-groundbreaking neuroscience.
It’s been tested on humans, but AFAIK no-one has ever felt compelled to go any further.
(Yeah, seems like it might be evidence. But then, maybe akrasia...)
Although it’s not technically possible yet, measuring the intensity of the positive and negative components of an experience sounds something that ought to be at least possible in principle.
I don’t see how having a quantitative, empirical measure which is appropriate for one individual helps you with comparisons across individuals. Do we really want to make people utility monsters because their neural currents devoted to measuring happiness have a higher amperage?
I was assuming that the measure would be valid across individuals. I wouldn’t expect the neural basis of suffering or pleasure to vary so much that you couldn’t automatically adapt it to the brains in question.
Do we really want to make people utility monsters because their neural currents devoted to measuring happiness have a higher amperage?
Well yes, hedonistic utilitarianism does make it possible in principle that Felix ends up screwing us over, but that’s an objection to hedonistic utilitarianism rather than the measure.
I was assuming that the measure would be valid across individuals.
I mean, the measure is going to be something like an EEG or an MRI, where we determine the amount of activity in some brain region. But while measuring the electrical properties of that region is just an engineering problem, and the units are the same from person to person, and maybe even the range is the same from person to person, that doesn’t establish the ethical principle that all people deserve equal consideration (or, in the case of range differences or variance differences, that neural activity determines how much consideration one deserves).
Well yes, hedonistic utilitarianism does make it possible in principle that Felix ends up screwing us over, but that’s an objection to hedonistic utilitarianism rather than the measure.
It’s not obvious to me that all agents deserve the same level of moral consideration (i.e. I am open to the possibility of utility monsters), but it is obvious to me that some ways of determining who should be the utility monsters are bad (generally because they’re easily hacked or provide unproductive incentives).
Well it’s not like people would go around maximizing the amount of this particular pattern of neural activity in the world: they would go around maximizing pleasure in the-kinds-of-agents-they-care-about, where the pattern is just a way of measuring and establishing what kinds of interventions actually do increase pleasure. (We are talking about humans, not FAI design, right?) If there are ways of hacking the pattern or producing it in ways that don’t actually correlate with pleasure (of the kind that we care about), then those can be identified and ignored.
Well it’s not like people would go around maximizing the amount of this particular pattern of neural activity in the world
Depending on your view of human psychology, this doesn’t seem like that bad a description, so long as we’re talking about people only maximizing their own circuitry. (Maximizing is probably wrong, rather than keeping it within some reference range.)
We are talking about humans, not FAI design, right?
That’s what I had that in mind, yeah.
My core objection, which I think lines up with SaidAchmiz’s, is that even if there’s the ability to measure people’s satisfaction objectively (so that we can count the transparency problem as solved), that doesn’t tell us how to make satisfaction tradeoffs between individuals.
even if there’s the ability to measure people’s satisfaction objectively (so that we can count the transparency problem as solved), that doesn’t tell us how to make satisfaction tradeoffs between individuals.
I agree with this. I was originally only objecting to the argument that aggregating utility between individuals would be impossible or incoherent, but I do not have an objection to the argument that the mapping from subjective states to math is underspecified. (Though I don’t see this as a serious problem for utilitarianism: it only means that different people will have different mappings rather than there being a single unique one.)
I was originally only objecting to the argument that aggregating utility between individuals would be impossible or incoherent
Er, hang on. If this is your objection, I’m not sure that you’ve actually said what’s wrong with said argument. Or do you mean that you were objecting to the applicability of said argument to hedonistic utilitarianism, which is how I read your comments?
To add to my “yes”: I agree with the claim that aggregating utility between individuals seems to be possibly incoherent in the context of preference utilitarianism. Indeed, if we define utility in terms of preferences, I’m even somewhat skeptical of the feasibility of optimizing the utility of a single individual over their lifetime: see this comment.
I approve of virtuous acts, and disapprove of vicious ones.
In terms of labels, I think I give consequentialist answers to the standard ethical questions, but I think most character improvement comes from thinking deontologically, because of the tremendous amount of influence our identities have on our actions. If one thinks of oneself as humble, that has many known ways of making one act differently. One’s abstract, far mode views are likely to only change one’s speech, not one’s behavior. Thus, I don’t put all that much effort into theories of ethics, and try to put effort instead into acting virtuously.
Interestingly, it seems our views are complementary, not contradictory. I would (I think) be willing to endorse what you said as a recipe for implementing the views I describe.
There is no such centralized place, no; I’ve alluded to my views in comments here and there over the past year or so, but haven’t gone laid them out fully. (Then again, I’m a member of no movements that depend heavily on any ethical positions. ;)
Truth be told — and I haven’t disguised this — my ethical views are not anywhere near completely fleshed-out. I know the general shape, I suppose, but beyond that I’m more sure about what I don’t believe — what objections and criticisms I have to other people’s views — than about what I do believe. But here’s a brief sketch.
I think that consequentialism, as a foundational idea, a basic approach, is the only one that makes sense. Deontology seems to me to be completely nonsensical as a grounding for ethics. Every seemingly-intelligent deontologist to whom I’ve spoken (which, admittedly, is a small number — a handful of people here in LessWrong) has appeared to be spouting utter nonsense. Deontology has its uses (see Bostrom’s “An Infinitarian Challenge to Aggregative Ethics”, and this post by Eliezer, for examples), but there it’s deployed for consequentialist reasons: we think it’ll give better results. I’ve seen the view expressed that virtue ethics is descriptively correct as an account of how human minds implement morality, and (as a result) prescriptively valid as a recommendation of how to implement your morality in your own mind once you’ve decided on your object-level moral views, and that seems like a more-or-less reasonable stance to take. As an actual philosophical grounding for morality, virtue ethics is nonsense, but perhaps that’s fine, given the above. Consequentialism actually makes sense. Consequences are the only things that matter? Well, yes. What else could there be?
As far as varieties of consequentialism go… I think intended and foreseeable consequences matter when evaluating the moral rightness of an act, not actual consequences; judging based on actual consequences seems utterly useless, because then you can’t even apply decision theory to the problem of deciding how to act. Judging on actual consequences also utterly fails to accord with my moral intuitions, while judging on intended and foreseeable consequences fits quite well.
I tend toward rule consequentialism rather than act consequentialism; I ask not “what would be the consequences of such an act?”, but “what sort of world would it be like, where [a suitably generalized class of] people acted in this [suitably generalized] way? Would I want to live in such a world?”, or something along those lines. I find act consequentialism to be too often short-sighted, and open to all sorts of dilemmas to which rule consequentialism simply does not fall prey.
I take seriously the complexity of value, and think that hedonistic utilitiarianism utterly fails to capture that complexity. I would not want to live in a world ruled by hedonistic utilitiarians. I wouldn’t want to hand them control of the future. I generally think that preferences are what’s important, and ought to be satisfied — I don’t think there’s any such thing as intrinsically immoral preferences (not even the preference to torture children), although of course one might have uninformed preferences (no, Mr. Example doesn’t really want to drink that glass of acid; what he wants is a glass of beer, and his apparent preference for acid would dissolve immediately, were he apprised of the facts); and satisfying certain preferences might introduce difficult conflicts (the fellow who wants to torture children — well, if satisfying his preferences would result in actual children being actually tortured, then I’m afraid we couldn’t have that). “I prefer to kill myself because I am depressed” is genuinely problematic, however. That’s an issue that I think about often.
All that seems like it might make me a preference utilitiarian, or something like it, but as I’ve said, I’m highly skeptical about the possibility or even coherence of aggregating utility across individuals, not to mention the fact that I don’t think my own preferences adhere to the VNM axioms, and so it may not even be possible to construct a utility function for all individuals. (The last person with whom I was discussing this stopped commenting on Lesswrong before I could get hold of my copy of Rational Choice in an Uncertain World, but now I’ve got it, and I’m willing to discuss the matter, if anyone likes.)
I don’t think it’s obvious that all beings that matter, matter equally. I don’t see anything wrong with valuing my mother much more than I value a randomly selected stranger in Mongolia. It’s not just that I do, in fact, value my mother more; I think it’s right that I should. My family and friends more than strangers; members of my culture (whatever that means, which isn’t necessarily “nation” or “country” or any such thing, though these things may be related) more than members of other cultures… this seems correct to me. (This seems to violate both the “equal consideration” and “agent-neutrality” aspects of classical utilitarianism, to again tie back to the SEP breakdown.)
As far as who matters — to a first approximation, I’d say it’s something like “beings intelligent and self-aware enough to consciously think about themselves”. Human-level intelligence and subjective consciousness, in other words. I don’t think animals matter. I don’t think unborn children matter, nor do infants (though there are nonetheless good reasons for not killing them, having to do with bright lines and so forth; similar considerations may protect the severely mentally disabled, though this is a matter which requires much further thought).
Do these thoughts add up to a coherent ethical system? Unlikely. They’re what I’ve got so far, though. Hopefully you find them at least somewhat useful, and of course feel free to ask me to elaborate, if you like.
Out of curiosity, what was your reason for asking about my ethical views in detail? I did somewhat enjoy writing out that comment, but I’m curious as to whether you were planning to go somewhere with this.
No big systematic overview, though several comments and posts of mine touch upon different parts of them. Is there anything in particular that you’re interested in?
If I could ask two quick questions, it’d be whether you’re a realist and whether you’re a cognitivist. The preponderance of those views within EA is what I’ve heard debated most often. (This is different from what first made me ask, but I’ll drop that.)
I know Jacy Anthis—thebestwecan on LessWrong—has an argument that realism combined with the moral beliefs about future generations typical among EAs suggests that smarter people in the future will work out a more correct ethics, and that this should significantly affect our actions now. He rejects realism, and think this is a bad consequence. I think it actually doesn’t depend on realism, but rather on most forms of cognitivism, for instance ones on which our coherent extrapolated view is correct. He plans to write about this.
Definitely not a realist. I haven’t looked at the exact definitions of these terms very much, but judging from the Wikipedia and SEP articles that I’ve skimmed, I’d call myself an ethical subjectivist (which apparently does fall under cognitivism).
I believe the prevalence of moral realism within EA is risky and bad for EA goals for several reasons. One of which is that moral realists tend to believe in the inevitability of a positive far-future (since smart minds will converge on the “right” morality), which tends to make them focus on ensuring the existence of the far future at the cost of other things.
If smart minds will converge on the “right” morality, this makes sense, but I severely doubt that is true. It could be true, but that possibility certainly isn’t worth sacrificing other goals of improvement.
And I think trying to figure out the “right” morality is a waste of resources for similar reasons. CEA has expressed the views I argue against here, which has other EAs and me concerned.
Can you suggest some? These could go into next year’s survey, though we’re keeping that short—more likely they’d go into a followup that Ben Landau-Taylor of Leverage Research is running.
I’d rather see ‘consequentialist’ supplemented or replaced by specific questions that get at substantive ethical or meta-ethical disputes in EA and philosophy. ‘Utilitarian’ and ‘deontologist’ mean lots of different things to different people, and on their strictest definitions they don’t entail a lot of their most interesting or widely cited ideas. Perhaps have an exploratory question one year asking non-utilitarians to write in their main objection to utilitarianism, then convert that into a series of questions the following year.
This was something I suggested to Tom because I’d be interested too. But ultimately we thought that only a small group of EAs would really have substantive ethical opinions and we thought to trim things for survey length. We added a box asking for clarifications at the end of the survey to provide more of this outlet.
One of the main objections to utilitarianism, it seems to me, is skepticism about the possibility (or even coherence of the notion) of aggregating utility across individuals. That’s one of my main objections, at any rate.
Skepticism about the applicability of the VNM theorem to human preferences is another issue, though that one might be less widespread.
Edit: The SEP describes classic utilitarianism as actual, direct, evaluative, hedonistic, maximizing, aggregative (specifically, total), universal, equal-consideration, agent-neutral consequentialism. I have definite issues with the “actual”, “direct”, “hedonistic”, “aggregative”, “total”, and “equal-consideration” parts of that. (Though I expect that my issues with “actual” will be shared by a significant portion of those who consider themselves utilitarians here, and my issues with “hedonistic” and “direct” may be as well. That leaves “aggregative”+”total”, and “equal-consideration”, as the two aspects most likely to be sources of philosophical conflict.)
Those sound like objections to preference utilitarianism but not hedonistic utilitarianism. Although it’s not technically possible yet, measuring the intensity of the positive and negative components of an experience sounds something that ought to be at least possible in principle. And the applicability of the VNM theorem to human preferences becomes irrelevant if you’re not interested in preferences in the first place.
Yes, true enough[1]; I did not properly separate those objections in my comment. To elaborate:
I object to hedonistic utilitarianism on the grounds that it clearly and grossly fails to capture my moral intuitions or those of anyone else whom I consider not to be evading the question. A full takedown of the “hedonistic” part of “hedonistic utilitarianism” is basically (at least) all of Eliezer’s posts about the complexity of value and so forth, and I won’t rehash it here.
To be honest, hedonistic utilitarianism seems to me to be so obviously wrong that I’m not even all that interested in having this sort of moral philosophy debate with an effective altruist (or anyone else) who holds such a view. I mean, to start with, my hypothetical interlocutor would have to rebut all the objections raised to hedonistic utilitarianism over the centuries since it’s been articulated, including, but not limited to, the aforementioned Lesswrong material.
I object to preference utilitarianism because of the “aggregation of utility” and “possibility of constructing a utility function” issues[2]. I think this is the more interesting objection.
[1] I’m not sure “intensity of the positive and negative components of an experience” is a coherent notion. There may not be a single quantity like that to measure. And even if we can measure something which we think qualifies for the title, it may be measurable only in some more-or-less absolute terms, while leaving open the question of how this hypothetical measured quantity matches up with anything like “utility to this particular experiencer”. But, for the sake of the argument, I’m willing to grant that such a quantity can indeed be usefully measured, because this is certainly not my true rejection.
[2] These are my objections to the “preference” component of preference utilitarianism; my objection to classical utilitarianism also includes objections to other components, which I have enumerated in the grandparent.
Two replies:
1) Even if hedonistic utilitarianism would ultimately be wrong as a full description of what a person values, “maximize pleasure while minimizing suffering” can still be a useful heuristic to follow. Yes, following that heuristic to its logical conclusion would mean forcibly rewiring everyone’s brains, but that doesn’t need to be a problem for as long as forcibly rewiring people’s brains isn’t a realistic option. HU may still be the best approximation of a person’s values in the context of today’s world, even if it wasn’t the best description overall.
2) The arguments on complexity of value and so on establish that the average person’s values aren’t correctly described by HU. This still leaves open the possibility of someone only approving of those of their behaviors that serve to promote HU, so there may definitely be individual people who accept HU, due to not sharing the moral intuitions which motivate the objections to it.
On 1): I am skeptical of replies to the effect that “yes, well, X might not be quite right, but it’s a useful heuristic, therefore I will go on acting as if X is right”. For one thing, a person who makes such a reply usually goes right back to saying “X is right!” (sans qualifiers) as soon as the current conversation ends. Let’s get clear on what we actually believe, I generally think; once we’ve firmly established that, we can look for maximally effective implementations.
For another thing, HU may be the best approximation etc. etc., but that’s a claim that at least should be made explicitly, such that it can be examined and argued for; a claim of this importance shouldn’t come up only in such tangential discussion branches.
For a third thing, what happens when forcibly rewiring people’s brains becomes a realistic option?
On 2): I think there’s two issues here. There could indeed be people who accept HU because that’s what correctly describes their moral intuitions. (Though I should certainly hope they do not think it proper to impose that moral philosophy on me, or on anyone else who doesn’t subscribe to HU!)
“Only approving of those behaviors that serve to promote HU” is, I think, a separate thing. Or at least, I’d need to see the concept expanded a bit more before I could judge. What does this hypothetical person believe? What moral intuitions do they have? What exactly does it mean to “promote” hedonistic utilitarianism?
Why would this be improper? Don’t that it doesn’t follow from any meta-ethical position.
If you say “all that matters is pain and pleasure”, and I say “no! I care about other things!”, and you’re like “nope, not listening. PAIN AND PLEASURE ARE THE ONLY THINGS”, and then proceed to enact policies which minimize pain and maximize pleasure, without regard for any of the other things that I care about, and all the while I’m telling you that no, I care about these other things! Stop ignoring them! Other things matter to me! but you’re not listening because you’ve decided that only pain and pleasure can possibly matter to anyone, despite my protestations otherwise...
… well, I hope you can see how that would bother me.
It’s not just a matter of us caring about different things. If it were only that, we could acknowledge the fact, and proceed to some sort of compromise. Hedonistic utilitiarians, however, do not acknowledge that it’s possible, or that it’s valid, to care about things that are not pain or pleasure. All these people who claim to care about all sorts of other things must be misguided! Clearly.
They may think it’s incorrect if they’re realists, or cognitivists of some other form. But this has nothing to do with their being HUs, only with their being cognitivists.
Here are 3 non-exhaustive ways in which the situation you described could be bothersome:
(i) If your first order ethical theory (as opposed to your meta-ethics), perhaps combined with very plausible facts about human nature, requires otherwise. For instance if it speaks in favour of toleration or liberty here.
(ii) If you’re a cognitivist of the sort who thinks she could be wrong, it could increase your credence that you’re wrong.
(iii) If you’d at least on reflection give weight to the evident distress SaidAchmiz feels in this scenario, as most HUs would.
No, I don’t think this is right. I think you (and Kaj_Sotala) are confusing these two questions:
Is it correct to hold an ethical view that is something other than hedonistic utilitarianism?
Does it make any sense to intrinsically value anything other than pleasure, or intrinsically disvalue things other than pain?
#1 is a meta-ethical question; moral realism or cognitivism may lead you to answer “no”, if you’re a hedonistic utilitarian. #2 is an ethical question; it’s about the content of hedonistic utilitarianism.
If I intrinsically care about, say, freedom, that’s not an ethical claim. It’s just a preference. “Humans may have preferences about things other than pain/pleasure, and those preferences are morally important” is an ethical claim which I might formulate, about that preference that I have.
Hedonistic utilitarianism tells me that my aforementioned preference is incoherent or mistaken, and that in fact I do not have any preferences (or any preferences that are morally important or worth caring about) other than preferences about pleasure/pain.
Moral realism (which, as blacktrance correctly notes, is implied by any utilitarianism) may lead a hedonistic utilitarian to say that my aforementioned ethical claim is incorrect.
As for your scenarios, I’m not sure what you meant by listing them. My point was that my scenario, which describes a situation involving a hypothetical me, Said Achmiz, would be bothersome to me, Said Achmiz. Is it really not clear why it would be?
Ethical subjectivism (which I subscribe to) would say that “ethical claims” are just a specific subset of our preferences; indeed, I’m rather skeptical of the notion of there being a distinction between ethical claims and preferences in the first place. But HU wouldn’t necessarily say that someone’s preference for something else than pleasure or pain would be mistaken—if it’s interpreted within a subjectivist framework, HU is just a description of preferences that are different. See my response to blacktrance.
I really don’t think that this is correct. If this were true, first of all, hedonistic utilitarianism would simply reduce to preference utilitarianism. In actual fact, neither view is merely about one’s own terminal values.
If someone, personally, cares only about pain and pleasure, but acknowledges that other people may have other things as terminal values, and thinks that The Good lies in satisfying everyone’s preferences maximally — which, for themselves, means maximizing pleasure and minimizing pain, and for other people may mean other things — then that person is not a hedonistic utilitarian. They are a preference utilitarian. Referring to them as an HU is simply not correct, because that’s not how the term is used in the philosophical literature.
On the other hand, if someone cares only about pain and pleasure — both theirs and other peoples’ — and would prefer that everyone’s pleasure be maximized and everyone’s pain be minimized; but this person is not a moral realist, and has no opinion on what constitutes The Good or thinks there’s no fact of the matter about whether an act is right or wrong; well, then this person is not a utilitarian at all. Again, describing this person as a hedonistic or any other kind of utilitarian completely fails to match up with how the term is used in the philosophical literature.
As for ethical subjectivism — uh, I don’t think that’s an actual thing. I’d not heard of anything by that name until today. I don’t like going by wikipedia’s definitions of philosophical principles, so I tried tracking it down to a source, such as perhaps a major philosopher espousing the view or at least describing it coherently. No such luck. Take a look at that list of references on its wikipedia page; two are to a single book (written in 1959 by some guy I’ve never heard of — have you? — and the shortness whose wikipedia page suggests that he wasn’t anyone interesting), and one is to a barely-related page that mentions the thing once, in passing, by a different name. I’m not convinced. As best I can tell, it’s a label that some modern-day historians of philosophy have used to describe… a not-quite-consistent family of views. (Divine command theory, for one.)
But let’s attempt to take it at face value. You say:
Very well. Are their attitudes correct, do they think? If they say there’s no fact of the matter about that, then they’re not a utilitiarian. “Utilitiarianism” is a quite established term in the literature. You can’t just apply it to any old thing.
Of course, this is Lesswrong; we don’t argue about definitions; we’re interested in what people actually think. However in this case I think getting our terms straight is important, for two reasons:
When most people say they’re utilitarians, they mean it in the usual sense, I think. So to understand what’s going on in these discussions, and in the heads of the people we’re talking to, we need to know what is the usual sense.
If you hold some view which is not one of the usual views with commonly-known terms, you shouldn’t call it by one of the commonly-known terms, because then I won’t have any idea what you’re talking about and we’ll keep getting into comment threads like this one.
… though, I just looked at the SEP entry on Consequentialism, and I note that aside for the title of one book in the bibliography, nowhere in the article is the word “realism” even mentioned. Nor does there seem to be an entry in the list of claims making up classic utilitarianism that would seem to require moral realism. I guess you could kind of interpret one of these three conditions as requiring moral realism:
… but it doesn’t seem obvious to me why someone who was both an ethical subjectivist couldn’t say that “I’m a classical utiliarian, in that (among other things) the best description of my ethical system is that I think that the goodness of an action should be determined based on how it affects all sentient beings, that benefits to one person matter just as much as similar benefits to others, and that the perspective of the people evaluating the consequences doesn’t matter. Though of course others could have ethical systems that were not well described by these items, and that wouldn’t make them wrong”.
Or maybe the important part in your comment was the part ”...but this person is not a moral realist, and has no opinion on what constitutes The Good”? But a subjectivist doesn’t say that he has no opinion on what constitutes The Good: he definitely has an opinion, and there may clearly be a right and wrong answer with regard to the kind of actions that are implied by his personal moral system; it’s just that the thing that constitutes The Good will be different for people with different moral systems.
Consequenialism supplies a realistic ontology, since it’s goods are facts about the real world, and utilitarian supplies an objective epistemology, since different utilitarians of the same stripe can converge. That adds up to some of the ingredients of realism, but not all of them. What is specifically lacking is an justification of comsequentialist ends as being objectively good, and not just subjectively desirable.
For this to make it realist, the fact that the truth of those facts has value would also have to be mind-independent. Even subjectivists typically value facts about the external world (e.g. their pleasure).
Ethical subjectivism is also discussed in the Stanford Encyclopedia of Philosophy.
(I like this quote from that article, btw: “So many debates in philosophy revolve around the issue of objectivity versus subjectivity that one may be forgiven for assuming that someone somewhere understands this distinction.”)
You may be right to say that my use of “utilitarian” is different from how it’s conventionally used in the literature; I’m pretty unfamiliar with the actual ethical literature. But if we have people who have the attitude of “I want to take the kinds of actions that maximally increase pleasure and maximally reduce suffering and I’m a moral realist” and people who have the attitude of “I want to take the kinds of actions that maximally increase pleasure and maximally reduce suffering and I’m a moral non-realist”, then it feels a little odd to have different terms for them, given that they probably have more in common with each other (with regard to the actions that they take and the views that they hold) than e.g. two people who are both moral realists but differ on consequentialism vs. deontology.
At least in a context where we are trying to categorize people into different camps based on what they think we should actually do, it would seem to make sense if we just called both the moral realist and moral non-realist “utilitarians”, if they both fit the description of a utilitarian otherwise.
I don’t think that hedonistic utilitarianism necessarily implies moral realism. Some HUs will certainly tell you that the people who morally disagree with them are misguided, but I don’t see why the proportion of HUs who think so (vs. the proportion of HUs who think that you are simply caring about different things) would need to be any different than it would be among the adherents of any other ethical position.
Maybe you meant your comment to refer specifically to the kinds of HUs who would impose their position on you, but even then the moral realism doesn’t follow. You can want to impose your values on others despite thinking that values are just questions of opinion. For instance, there are things that I consider basic human rights and I want to impose the requirement to respect them on every member of every society, even though there are people who would disagree with that requirement. I don’t think that the people who disagree are misguided in any sense, I just think that they value different things.
I agree with blacktrance’s reply to you, and also see my reply to tog in a different subthread for some commentary. However, I’m sufficiently unsure of what you’re saying to be certain that your comment is fully answered by either of those things. For example:
If you [the hypothetical you] think that it’s possible to care (intrinsically, i.e. terminally) about things other than pain and pleasure, then I’m not quite sure how you can remain a hedonistic utilitarian. You’d have to say something like: “Yes, many people intrinsically value all sorts of things, but those preferences are morally irrelevant, and it is ok to frustrate those preferences as much as necessary, in order to minimize pain and maximize pleasure.” You would, in other words, have to endorse a world where all the things that people value are mercilessly destroyed, and the things they most abhor and despise come to pass, if only this world had the most pleasure and least pain.
Now, granted, people sometimes endorse the strangest things, and I wouldn’t even be surprised to find someone on Lesswrong who held such a view, but then again I never claimed otherwise. What I said was that I should hope those people do not impose such a worldview on me.
If I’ve misinterpreted your comment and thereby failed to address your points, apologies; please clarify.
Well, if you’re really curious about how one could be a hedonistic utilitarian while also thinking that it’s possible to care intrinsically about things other than pain and pleasure, one could think something like:
“So there’s this confusing concept called ‘preferences’ that seems to be a general term for all kinds of things that affect our behavior, or mental states, or both. Probably not all the things that affect our behavior are morally important: for instance, a reflex action is a thing in a person’s nervous system that causes them to act in a certain way in certain situations, so you could kind of call that a preference to act in such a way in such a situation, but it still doesn’t seem like a morally important one.
“So what does make a preference morally important? If we define a preference as ‘an internal disposition that affects the choices that you make’, it seems like there would exist two kinds of preferences. First there are the ones that just cause a person to do things, but which don’t necessarily cause any feelings of pleasure or pain. Reflexes and automated habits, for instance. These don’t feel like they’d be worth moral consideration any more than the automatic decisions made by a computer program would.
“But then there’s the second category of preferences, ones that cause pleasure when they are satisfied, suffering when they are frustrated, or both. It feel like pleasure is a good thing and suffering is a bad thing, so that makes it good to satisfy the kinds of preferences that are produce pleasure when satisfied, as well as bad to frustrate the kinds of preferences that cause suffering when frustrated. Aha! Now I seem to have found a reasonable guideline for the kinds of preferences that I should care about. And of course this goes for higher-order preferences as well: if someone cares about X, then trying to change that preference would be a bad thing if they had a preference to continue caring about X, such that they would feel bad if someone tried to change their caring about X.
“And of course people can have various intrinsic preferences for things, which can mean that they do things even though that doesn’t produce them any suffering or pleasure. Or it can mean that doing something gives them pleasure or lets them avoid suffering by itself, even when doing that something doesn’t lead to any other consequence. The first kind of intrinsic preference I already concluded was morally irrelevant; the second kind is worth respecting, again because violating it would cause suffering, or reduce pleasure, or both. And I get tired of saying something clumsy like ‘increasing pleasure and decreasing suffering’ all the time, so let’s just call that ‘increasing well-being’ for short.
“Now unfortunately people have lots of different intrinsic preferences, and they often conflict. We can’t satisfy them all, as nice as it would be, so I have to choose my side. Since I chose my favored preferences on the basis that pleasure is good and suffering is bad, it would make sense to side with the preferences that, in the long term, produce the greatest amount of well-being in the world. For instance, some people may want the freedom to lie and cheat and murder, whereas other people want to have a peaceful and well-organized society. I think the preferences for living in peace will lead to greater well-being in the long term, so I will side with them, even if that means that the preferences of the sociopaths and murderers will be frustrated.
“Now there’s also this kind of inconvenient issue that if we rewire people’s brains so that they’ll always experience the maximal amount of pleasure, then that will produce more well-being in the long run, even if those people don’t currently want to have their brains rewired. I previously concluded that I should side with kinds of preferences that produce the greatest amount of well-being in the world, and the preference of ‘let’s rewire everyone’s brains’ does seem to produce by far the greatest amount of well-being in the world. So I should side with that preference, even though it goes against the intrinsic preferences of a lot of other people, but so did the decision to impose a lawful and peaceful society on the sociopaths and murderers, so that’s okay by me.
“Of course, other people may disagree, since they care about different things than pain and pleasure. And they’re not any more or less right—they just have different criteria for what counts as a moral action. But if it’s either them imposing their worldview on me, or me imposing my worldview on them, well, I’d rather have it be me imposing mine on them.”
Right, I wasn’t objecting to your statement of not wanting to have such a worldview imposed on you. I was only objecting to the statement that hedonistic utilitarians would necessarily have to think that others were misguided in some sense.
Any form of utilitarianism implies moral realism, as utilitarianism is a normative ethical theory and normative ethical theories presuppose moral realism.
I feel that this discussion is rapidly descending into a debate over definitions, but as a counter-example, take ethical subjectivism, which is a form of moral non-realism and which Wikipedia defines as claiming that:
Someone could be an ethical subjectivist and say that utilitarianism is the theory that best describes their particular attitudes, or at least that subset of their attitudes that they endorse.
Someone could be an ethical subjectivist and want to maximize world utility, but such a person would not be a utilitarian, because utilitarianism holds that other people should maximize world utility. If you merely say “I want to maximize world utility and others to do the same”, that is not utilitarianism—a utilitarian would say that you ought to maximize world utility, even if you don’t want to, and it’s not a matter of attitudes. Yes, this is arguing over definitions to some extent, but it’s important because I often see this kind of confusion about utilitarianism on LW.
Could you provide a reference for that? At least the SEP entry on the topic doesn’t clearly state this. I’m also unsure of what difference this makes in practice—I guess we could come up with a new word for all the people who are both moral antirealist and utilitarian-aside-for-being-moral-antirealists, but I’m not sure if the difference in their behavior and beliefs is large enough for that to be worth it.
Non egoistic subjectivists?
The SEP entry for consequentialism says it “is the view that normative properties depend only on consequences”, implying a belief in normative properties, which means moral realism.
If you want to describe people’s actions, a utilitarian and a world-utility-maximizing non-realist would act similarly, but there would be differences in attitude: a utilitarian would say and feel like he is doing the morally right thing and those who disagree with him are in error, whereas the non-realist would merely feel like he is doing what he wants and that there is nothing special about wanting to maximize world utility—to him, it’s just another preference, like collecting stamps or eating ice cream.
This is getting way too much into a debate over definitions so I’ll stop after this comment, but I’ll just point out that, among professional philosophers, there is no correlation between endorsing consequentialism and endorsing moral realism.
A non-consequentialist could be a moral realist as well, such as if they were a deontologist, so it’s not a good measurement.
Also, consequentialism and moral realism aren’t always well-defined terms.
Edit: That survey’s results are strange. Twenty people answered that they’re moral realists but non-cognitivists, though moral realism is necessarily cognitivist.
That doesn’t mean utilitarianism is subjective. Rather, it means any subjective idea could correspond to objective truth.
I agree that it would often be good to be clearer about these points.
At that point the people who consider themselves hedonistic utilitarians might come up with a theory that says that forcible wireheading is wrong and switch to calling themselves supporters of that theory. Or they could go on calling themselves HUs despite not forcibly wireheading anyone, in the same way that many people call themselves utilitarians today despite not actually giving most of their income away. Or some of them could decide to start working towards efforts to forcibly wirehead everyone, in which case they’d become the kinds of people described by my reply 2).
By this, I meant to say “only approve of whatever course of action HU says is the best one”.
Yeah, I meant that as a normative “what then”, not an empirical one. I agree that what you describe are plausible scenarios.
In that case, I’m unsure of what kind of an answer you were expecting (unless the “what then” was meant as a rhetorical question, but even then I’m slightly unsure of what point it was making).
Yes, the “what then” was rhetorical. If I had to express my point non-rhetorically, it’d be something like this:
If you take a position which gives ethically correct results only until such time as some (reasonably plausible) scenario comes to pass, then maybe your position isn’t ethical in the first place. “This ethical framework gives nonsensical or monstrous results in edge cases [of varying degrees of edge-ness]” is, after all, a common and quite justified criticism of ethical frameworks.
It is a point against the framework, certainly. But so far nobody has developed an ethical framework that would have no problems at all, so at the moment we can only choose the framework that’s the least bad.
(Assuming that we wish to choose one in the first place, of course—I do think that there is merit in just accepting that they’re all flawed and then not choosing to endorse any single one.)
Well, that’s been my policy so far, certainly. Some are worse than others, though. “This ethical framework breaks in catastrophic, horrifying fashion, creating an instant dystopia, as soon as we can rewire people’s brains” is pretty darn bad.
… can’t we rewire brains right now? We just … don’t.
Well, we must not be hedonistic utilitarians then, right? Because if we were, and we could, we would.
Edit: Also, what the heck are you talking about?
Wireheading. The term is not a metaphor, and it’s not a hypothetical. You can literally stick a wire into someone’s pleasure centers and activate them, using only non-groundbreaking neuroscience.
It’s been tested on humans, but AFAIK no-one has ever felt compelled to go any further.
(Yeah, seems like it might be evidence. But then, maybe akrasia...)
Where and what are these “pleasure centers”, exactly?
I don’t see how having a quantitative, empirical measure which is appropriate for one individual helps you with comparisons across individuals. Do we really want to make people utility monsters because their neural currents devoted to measuring happiness have a higher amperage?
I was assuming that the measure would be valid across individuals. I wouldn’t expect the neural basis of suffering or pleasure to vary so much that you couldn’t automatically adapt it to the brains in question.
Well yes, hedonistic utilitarianism does make it possible in principle that Felix ends up screwing us over, but that’s an objection to hedonistic utilitarianism rather than the measure.
I mean, the measure is going to be something like an EEG or an MRI, where we determine the amount of activity in some brain region. But while measuring the electrical properties of that region is just an engineering problem, and the units are the same from person to person, and maybe even the range is the same from person to person, that doesn’t establish the ethical principle that all people deserve equal consideration (or, in the case of range differences or variance differences, that neural activity determines how much consideration one deserves).
It’s not obvious to me that all agents deserve the same level of moral consideration (i.e. I am open to the possibility of utility monsters), but it is obvious to me that some ways of determining who should be the utility monsters are bad (generally because they’re easily hacked or provide unproductive incentives).
Well it’s not like people would go around maximizing the amount of this particular pattern of neural activity in the world: they would go around maximizing pleasure in the-kinds-of-agents-they-care-about, where the pattern is just a way of measuring and establishing what kinds of interventions actually do increase pleasure. (We are talking about humans, not FAI design, right?) If there are ways of hacking the pattern or producing it in ways that don’t actually correlate with pleasure (of the kind that we care about), then those can be identified and ignored.
Depending on your view of human psychology, this doesn’t seem like that bad a description, so long as we’re talking about people only maximizing their own circuitry. (Maximizing is probably wrong, rather than keeping it within some reference range.)
That’s what I had that in mind, yeah.
My core objection, which I think lines up with SaidAchmiz’s, is that even if there’s the ability to measure people’s satisfaction objectively (so that we can count the transparency problem as solved), that doesn’t tell us how to make satisfaction tradeoffs between individuals.
I agree with this. I was originally only objecting to the argument that aggregating utility between individuals would be impossible or incoherent, but I do not have an objection to the argument that the mapping from subjective states to math is underspecified. (Though I don’t see this as a serious problem for utilitarianism: it only means that different people will have different mappings rather than there being a single unique one.)
Er, hang on. If this is your objection, I’m not sure that you’ve actually said what’s wrong with said argument. Or do you mean that you were objecting to the applicability of said argument to hedonistic utilitarianism, which is how I read your comments?
To add to my “yes”: I agree with the claim that aggregating utility between individuals seems to be possibly incoherent in the context of preference utilitarianism. Indeed, if we define utility in terms of preferences, I’m even somewhat skeptical of the feasibility of optimizing the utility of a single individual over their lifetime: see this comment.
Yes.
Kaj, is there somewhere you lay out your ethical views in more detail?
Ditto for Vaniver and Said.
I approve of virtuous acts, and disapprove of vicious ones.
In terms of labels, I think I give consequentialist answers to the standard ethical questions, but I think most character improvement comes from thinking deontologically, because of the tremendous amount of influence our identities have on our actions. If one thinks of oneself as humble, that has many known ways of making one act differently. One’s abstract, far mode views are likely to only change one’s speech, not one’s behavior. Thus, I don’t put all that much effort into theories of ethics, and try to put effort instead into acting virtuously.
Interestingly, it seems our views are complementary, not contradictory. I would (I think) be willing to endorse what you said as a recipe for implementing the views I describe.
There is no such centralized place, no; I’ve alluded to my views in comments here and there over the past year or so, but haven’t gone laid them out fully. (Then again, I’m a member of no movements that depend heavily on any ethical positions. ;)
Truth be told — and I haven’t disguised this — my ethical views are not anywhere near completely fleshed-out. I know the general shape, I suppose, but beyond that I’m more sure about what I don’t believe — what objections and criticisms I have to other people’s views — than about what I do believe. But here’s a brief sketch.
I think that consequentialism, as a foundational idea, a basic approach, is the only one that makes sense. Deontology seems to me to be completely nonsensical as a grounding for ethics. Every seemingly-intelligent deontologist to whom I’ve spoken (which, admittedly, is a small number — a handful of people here in LessWrong) has appeared to be spouting utter nonsense. Deontology has its uses (see Bostrom’s “An Infinitarian Challenge to Aggregative Ethics”, and this post by Eliezer, for examples), but there it’s deployed for consequentialist reasons: we think it’ll give better results. I’ve seen the view expressed that virtue ethics is descriptively correct as an account of how human minds implement morality, and (as a result) prescriptively valid as a recommendation of how to implement your morality in your own mind once you’ve decided on your object-level moral views, and that seems like a more-or-less reasonable stance to take. As an actual philosophical grounding for morality, virtue ethics is nonsense, but perhaps that’s fine, given the above. Consequentialism actually makes sense. Consequences are the only things that matter? Well, yes. What else could there be?
As far as varieties of consequentialism go… I think intended and foreseeable consequences matter when evaluating the moral rightness of an act, not actual consequences; judging based on actual consequences seems utterly useless, because then you can’t even apply decision theory to the problem of deciding how to act. Judging on actual consequences also utterly fails to accord with my moral intuitions, while judging on intended and foreseeable consequences fits quite well.
I tend toward rule consequentialism rather than act consequentialism; I ask not “what would be the consequences of such an act?”, but “what sort of world would it be like, where [a suitably generalized class of] people acted in this [suitably generalized] way? Would I want to live in such a world?”, or something along those lines. I find act consequentialism to be too often short-sighted, and open to all sorts of dilemmas to which rule consequentialism simply does not fall prey.
I take seriously the complexity of value, and think that hedonistic utilitiarianism utterly fails to capture that complexity. I would not want to live in a world ruled by hedonistic utilitiarians. I wouldn’t want to hand them control of the future. I generally think that preferences are what’s important, and ought to be satisfied — I don’t think there’s any such thing as intrinsically immoral preferences (not even the preference to torture children), although of course one might have uninformed preferences (no, Mr. Example doesn’t really want to drink that glass of acid; what he wants is a glass of beer, and his apparent preference for acid would dissolve immediately, were he apprised of the facts); and satisfying certain preferences might introduce difficult conflicts (the fellow who wants to torture children — well, if satisfying his preferences would result in actual children being actually tortured, then I’m afraid we couldn’t have that). “I prefer to kill myself because I am depressed” is genuinely problematic, however. That’s an issue that I think about often.
All that seems like it might make me a preference utilitiarian, or something like it, but as I’ve said, I’m highly skeptical about the possibility or even coherence of aggregating utility across individuals, not to mention the fact that I don’t think my own preferences adhere to the VNM axioms, and so it may not even be possible to construct a utility function for all individuals. (The last person with whom I was discussing this stopped commenting on Lesswrong before I could get hold of my copy of Rational Choice in an Uncertain World, but now I’ve got it, and I’m willing to discuss the matter, if anyone likes.)
I don’t think it’s obvious that all beings that matter, matter equally. I don’t see anything wrong with valuing my mother much more than I value a randomly selected stranger in Mongolia. It’s not just that I do, in fact, value my mother more; I think it’s right that I should. My family and friends more than strangers; members of my culture (whatever that means, which isn’t necessarily “nation” or “country” or any such thing, though these things may be related) more than members of other cultures… this seems correct to me. (This seems to violate both the “equal consideration” and “agent-neutrality” aspects of classical utilitarianism, to again tie back to the SEP breakdown.)
As far as who matters — to a first approximation, I’d say it’s something like “beings intelligent and self-aware enough to consciously think about themselves”. Human-level intelligence and subjective consciousness, in other words. I don’t think animals matter. I don’t think unborn children matter, nor do infants (though there are nonetheless good reasons for not killing them, having to do with bright lines and so forth; similar considerations may protect the severely mentally disabled, though this is a matter which requires much further thought).
Do these thoughts add up to a coherent ethical system? Unlikely. They’re what I’ve got so far, though. Hopefully you find them at least somewhat useful, and of course feel free to ask me to elaborate, if you like.
Out of curiosity, what was your reason for asking about my ethical views in detail? I did somewhat enjoy writing out that comment, but I’m curious as to whether you were planning to go somewhere with this.
I’m glad you enjoyed it, as you’re right I didn’t go anywhere—I got distracted by other thing. But it was partly a sort of straw poll to supplement the survey, and partly connected to these concerns: http://lesswrong.com/lw/k60/2014_survey_of_effective_altruists/aw1p
No big systematic overview, though several comments and posts of mine touch upon different parts of them. Is there anything in particular that you’re interested in?
If I could ask two quick questions, it’d be whether you’re a realist and whether you’re a cognitivist. The preponderance of those views within EA is what I’ve heard debated most often. (This is different from what first made me ask, but I’ll drop that.)
I know Jacy Anthis—thebestwecan on LessWrong—has an argument that realism combined with the moral beliefs about future generations typical among EAs suggests that smarter people in the future will work out a more correct ethics, and that this should significantly affect our actions now. He rejects realism, and think this is a bad consequence. I think it actually doesn’t depend on realism, but rather on most forms of cognitivism, for instance ones on which our coherent extrapolated view is correct. He plans to write about this.
Definitely not a realist. I haven’t looked at the exact definitions of these terms very much, but judging from the Wikipedia and SEP articles that I’ve skimmed, I’d call myself an ethical subjectivist (which apparently does fall under cognitivism).
I believe the prevalence of moral realism within EA is risky and bad for EA goals for several reasons. One of which is that moral realists tend to believe in the inevitability of a positive far-future (since smart minds will converge on the “right” morality), which tends to make them focus on ensuring the existence of the far future at the cost of other things.
If smart minds will converge on the “right” morality, this makes sense, but I severely doubt that is true. It could be true, but that possibility certainly isn’t worth sacrificing other goals of improvement.
And I think trying to figure out the “right” morality is a waste of resources for similar reasons. CEA has expressed the views I argue against here, which has other EAs and me concerned.
Can you suggest some? These could go into next year’s survey, though we’re keeping that short—more likely they’d go into a followup that Ben Landau-Taylor of Leverage Research is running.