What irritates me about this post is that Yudkowsky just seems to assume without questioning (at least not in that article and related ones) that we ought be concerned about human morality. In “Fake Utility Functions”, he argues that hedonistic utilitarianism fails to due justice to all the complex human values . But that’s not the goal utiltiarians wanted to achieve, that’s not their view of ethics. Ethics should be independent of the evolutionary psychology of Homo sapiens. Self-aware beings could have ended up with different values. What are the meta criteria by which we should decide what values to have in the first place? Hedonistic utilitarians answer that what matters, ultimately, can only be conscious experience. Yudkowsky seemed to assume that hedonistic utiltiarians thought that humans must want to be hedonistic utiltiarians deep down. But they don’t need that to be the case at all. Human ethical intuitions could well be more misguided than Yudkowsky acknowledges anyway (i.e. that many people have strong intuitions against some of the consequences of consequentialism). Yudkowsky’s dismissal of the One Great Moral Principle thus seems hastened. Toby Ord made a similar point in the comments to “Fake Utility Functions”. (I don’t want to advocate classical utilitarianism here because I think there are reasons that speak against happiness being the relevant criterion, I just wanted to point out that more thought should be given to this foundational issue of ethics.)
Lukas_Gloor
If there remains a non-zero probability of the proof being wrong, I’d carry on with my ethics, of course.
But if that’s ruled out, there would be no reason to favor any one action over any other action. Still, I don’t think I’d ever torture myself, even though the hypothetical postulates that this would be a pointless aversion. I’d have to admit that I have no good reasons for not doing something else, yet I’d go the path of the least resistance with maybe some short-term happy activities (i.e. eating tasty animal products) and then suicide in a relatively painless way. I have no intuition that non-existence is worse than happiness, and I would still find my aversion to suffering motivating enough to determine the choice of action.
Total utilitarianism is defined as maximising the sum of everyone’s individual utility function.
That seems misleading. Most of the time “total utiltiarianism” refers to what should actually be called “hedonistic total utilitarianism”. And what is maximized there is the suprlus of happiness over suffering (positive hedonic states over negative ones), which isn’t necessarily synonymous with individual utility functions.
There are three different parameters for the various kinds of utilitarianism: It can either be total or average or prior-existence. Then it can be negative or classical (and in theory also “positive”, even though that would be insane, forcing people to accept eternal torture if there’s even the slightest chance of a moment of happiness). And then utiltiarianism can also be hedonistic or preference. Most common, and subject to this article, is (classical) total hedonistic utiltiarianism. While some combinations make very little sense, a lot of them actually have advocates. (For instance, recently someone published a paper advocating “negative average preference-utilitarianism”.)
I know total utilitarians who’d have no problem with that. Imagine simulated minds instead of carbon-based ones. If you can just imagine shutting one simulation off and turning on another one, this can eliminate some of our intuitive aversions to killing and maybe it will make the conclusion less counterintuitive. Personally I’m not a total utilitarian, but I don’t think that’s a particularly problematic aspect of it.
My problem with total hedonistic utiltiarianism is the following: Imagine a planet full of beings living in terrible suffering. You have the choice to either euthanize them all (or just make them happy), or let them go on living forever, while also creating a sufficiently huge number of beings with lives barely worth living somewhere else. Now that I find unacceptable. I don’t think you do anything good by bringing a happy being into existence.
What seems to be overlooked in most discussions about total hedonistic utiltiarianism is that the proponents often have a specific (Parfitean) view about personal identity. Which leads to either empty or open individualism. Based on that, they may hold that it is no more rational to care about one’s own future self than it is to care about any other future self. “Killing” a being would then just be failing to let a new moment of consciousness come into existence. And any notions of “preferences” would not really make sense anymore, only instrumentally.
- 28 Jun 2012 2:46 UTC; 1 point) 's comment on A (small) critique of total utilitarianism by (
- 26 Jun 2012 20:53 UTC; -2 points) 's comment on A (small) critique of total utilitarianism by (
But isn’t the “values of individual lives” preference-utilitarianism (which often comes as prior-existence instead of “total”)? I’m confused, it seems like there are several definitions criculating. I haven’t encountered this kind of total utilitarianism on the felicifia utilitarianism forum. The quoted conclusion about killing people and replacing people is accurate, according the definition that is familiar to me.
Sounds like total preference-utiltiarianism, instead of total hedonistic utilitarianism. Would this view imply that it is good to create beings whose preferences are satisfied? If yes, then it’s total PU. If no, then it might be prior-existence PU. The original article doesn’t specify explicitly whether it means hedonistic or preference utilitarianism, but the example given about killing only works for hedonistic utilitarianism, that’s why I assumed that this is what’s meant. However, somewhere else in the article, it says
Total utilitarianism is defined as maximising the sum of everyone’s individual utility function.
And that seems more like preference-utilitarianism again. So something doesn’t work out here.
As a side note, I’ve actually never encountered a total preference-utilitarian, only prior-existence ones (like Peter Singer). But it’s a consistent position.
Whatever the piece assumes, I don’t think it’s preference utilitarianism because then the first sentence doesn’t make sense:
In total utilitarianism, it is a morally neutral act to kill someone (in a painless and unexpected manner) and creating/giving birth to another being of comparable happiness.
Assuming most people have a preference to go on living, as well as various other preferences for the future, then killing them would violate all these preferences, and simply creating a new, equally happy being would still leave you with less overall utility, because all the unsatisfied preferences count negatively. (Or is there a version of preference utilitarianism where unsatisfied preferences don’t count negatively?) The being would have to be substantially happier, or you’d need a lot more beings to make up for the unsatisfied preferences caused by the killing. Unless we’re talking about beings that live “in the moment”, where their preferences correspond to momentary hedonism.
Peter Singer wrote a chapter on killing and replaceability in Practical Ethics. His view is prior-existence, not total preference utilitarianism, but the points on replaceability apply to both.
I see, interesting. That means you bring in a notion independent of both the person’s experiences and preferences. You bring in a particular view on value (e.g. that life shouldn’t be repetitious). I’d just call this a consequentialist theory where the exact values would have to be specified in the description, instead of utilitarianism. But that’s just semantics, as you said initially, it’s important that we specify what exactly we’re talking about.
Yes. The error is that humans aren’t good at utilitarianism.
Why would that be an error? It’s not a requirement for an ethical theory that Homo sapiens must be good at it. If we notice that humans are bad at it, maybe we should make AI or posthumans that are better at it, if we truly view this as the best ethical theory. Besides, if the outcome of people following utilitarianism is really that bad, then utilitarianism would demand (it gets meta now) that people should follow some other theory that overall has better outcomes (see also Parfit’s Reasons and Persons). Another solution is Hare’s proposed “Two-Level Utilitarianism”. From Wikipedia:
Hare proposed that on a day to day basis, one should think and act like a rule utilitarian and follow a set of intuitive prima facie rules, in order to avoid human error and bias influencing one’s decision-making, and thus avoiding the problems that affected act utilitarianism.
This, in my opinion, is by itself a decisive argument against utilitarianism.
You mean against preference-utilitarianism.
The vast majority of utilitarians I know are hedonistic utilitarians, where this criticism doesn’t apply at all. (For some reason LW seems to be totally focused on preference-utilitarianism, as I’ve noticed by now.) As for the criticism itself: I agree! Preference-utiltiarians can come up with sensible estimates and intuitive judgements, but when you actually try to show that in theory there is one right answer, you just find a huge mess.
Yes but it doesn’t have the problem Vladimir_M described above, and it can bite the bullet in the repugnant conclusion by appealing to personal identity being an illusion. Total hedonistic utilitarianism is quite hard to argue against, actually.
Hedonistic utilitarianism is not about preferences at all. It’s about maximizing happiness, whatever the reason or substrate for it. The utilitronium shockwave would be the best scenario for total hedonistic utilitarianism.
Good catch, I’m well aware of that. I didn’t say that I think bringing a neutral being into existence is neutral. If the neutral being’s life contains suffering, then the suffering counts negatively. Prior-existence views seem to not work without the inconsistency you pointed out. The only consistent alternative to total utiltiarianism is, as I see it currently, negative utilitarianism. Which has its own repugnant conclusions (e.g. anti-natalism), but for several reasons I find those easier to accept.
No, nothing of that sort. You just take the surplus of positive hedonic states over negative ones and try to maximize that. Interpersonal boundaries become irrelevant, in fact many hedonistic utilitarians think that the concept of personal identity is an illusion anyway. If you consider utility functions, then that’s preference utilitarianism or something else entirely.
Is intrapersonal comparison possible? Personal boundaries don’t matter for hedonistic utilitarianism, they only matter insofar as you may have spatio-temporally connected clusters of hedons (lives). The difficulties in comparison seem to be of an empirical nature, not a fundamental one (unlike the problems with preference-utilitarianism). If we had a good enough theory of consciousness, we could quantitatively describe the possible states of consciousness and their hedonic tones. Or not?
One common argument against hedonistic utiltiarianism is that there are “different kinds of pleasures”, and that they are “incommensurable”. But if that we’re the case, it would be irrational to accept a trade-off of the lowest pleasure of one sort for the highest pleasure of another sort, and no one would actually claim that. So even if pleasures “differ in kind”, there’d be an empirical trade-off value based on how pleasant the hedonic states actually are.
I should qualify my statement. I was talking only about the common varieties of utilitarianism and I may well have omitted consistent variants that are unpopular or weird (e.g. something like negative average preference-utilitarianism). Basically my point was that “hybrid-views” like prior-existence (or “critical level” negative utiltiarianism) run into contradictions. Most forms of average utilitarianism aren’t contradictory, but they imply an obvious absurdity: A world with one being in maximum suffering would be [edit:] worse than a world with a billion beings in suffering that’s just slightly less awful.
Not in the formal sense. I meant for instance what Will_Savin pointed out above, a neutral life (a lot of suffering and a lot of happiness) being equally worthy of creating as a happy one (mainly just happiness, very little suffering). Or for “critical levels” (which also refers to the infamous dust specks), see section VI of this paper, where you get different results depending on how you start aggregating. And Peter Singer’s prior-existence view seems to contain a “contradiction” (maybe “absurdity” is better) as well having to do with replaceability, but that would take me a while to explain. It’s not quite a contradiction that the theory states “do X and not-X”, but it’s obvious enough that something doesn’t add up. I hope that led to some clarification, sorry for my terminology.
Indeed, thanks.
Hi! I discovered LW about a year ago and now I actually created an account. I study philosophy, and biology as minor. Sometimes I’m rather shocked by the things my fellow students believe and how they argue for their beliefs; I wish something like LW would be part of the standard curriculum. My main interests are ethics, philosophy of mind and evolutionary biology, and I’m looking forward to participating in discussions on these issues. Especially on ethics, as I’m skeptical regarding some of the views advocated on here (I’m a utilitarian). As someone who had read the original books several times, I was also delighted to find out about HPMoR recently.