Rob Bensinger
Karma: 8,129 (LW), 15 (AF)
NewTop
The above updates me toward being more uncertain about whether it’s a good idea to add an ‘optional non-anonymized upvoting’ feature. I’ll note that separating out ‘I agree with this’ from ‘I want to see more comments like this’ is potentially extra valuable (maybe even necessary) for a healthy non-anonymized upvoting system, because it’s more important to distinguish those things if your name’s on the line. Also, non-anonymized ‘I factually disagree with this’ is a lot more useful than non-anonymized ‘I want to see fewer comments/posts like this’.
The utility of others is not my utility, therefore I am not a utilitarian. I reject unconditional altruism in general for this reason.
When I say that I’m a utilitarian (or something utilitarian-ish), I mean something like: If there were no non-obvious bad side-effects — e.g., it doesn’t damage my ability to have ordinary human relationships in a way that ends up burning more value than it creates — I’d take a pill that would bind my future self to be unwilling to sacrifice two strangers to save a friend (or to save myself), all else being equal.
The not-obviously-confused-or-silly version of utilitarianism is “not reflectively endorsing extreme partiality toward yourself or your friends relative to strangers,” rather than “I literally have no goals or preferences or affection for anything other than perfectly unbiased maximization of everyone’s welfare’“.
Long articles are often easier to refute because they make more claims, and their claims are more detailed.
Additionally, the point of writing a blog post isn’t to make it easy to refute; and you don’t get extra points for refuting an entire post vs. a piece of a post.
And if you say “I don’t push the button, but only because I want to cooperate with other moral theorists” or “I don’t push the button, but only because NU is very very likely true but I have nonzero moral uncertainty”: do you really think that’s the reason? Does that really sound like the prescription of the correct normative theory (modulo your own cognitive limitations and resultant moral uncertainty)? If the negotiation-between-moral-theories spat out a slightly different answer, would this actually be a good idea?
In evolutionary and developmental history terms, we can see at the first quick glance that many (if not immediately all) of our other motivations interact with suffering, or have interacted with our suffering in the past (individually, neurally, culturally, evolutionarily). They serve functions of group cohesion, coping with stress, acquiring resources, intimacy, adaptive learning & growth, social deterrence, self-protection, understanding ourselves, and various other things we value & honor because they make life easier or interesting.
Seems like all of this could also be said of things like “preferences”, “enjoyment”, “satisfaction”, “feelings of correctness”, “attention”, “awareness”, “imagination”, “social modeling”, “surprise”, “planning”, “coordination”, “memory”, “variety”, “novelty”, and many other things.
“Preferences” in particular seems like an obvious candidate for ‘thing to reduce morality to’; what’s your argument for only basing our decisions on dispreference or displeasure and ignoring positive preferences or pleasure (except instrumentally)?
Neither NU nor other systems will honor all of our perceived wants as absolutes to maximize
I’m not sure I understand your argument here. Yes, values are complicated and can conflict with each other. But I’d rather try to find reasonable-though-imperfect approximations and tradeoffs, rather than pick a utility function I know doesn’t match human values and optimize it instead just because it’s uncomplicated and lets us off the hook for thinking about tradeoffs between things we ultimately care about.
E.g., I like pizza. You could say that it’s hard to list every possible flavor I enjoy in perfect detail and completeness, but I’m not thereby tempted to stop eating pizza, or to try to reduce my pizza desire to some other goal like ‘existential risk minimization’ or ‘suffering minimization’. Pizza is just one of the things I like.
To actually reject NU, you must explain what makes something (other than suffering) terminally valuable (or as I say, motivating) beyond its instrumental value for helping us prevent suffering in the total context
E.g.: I enjoy it. If my friends have more fun watching action movies than rom-coms, then I’ll happily say that that’s sufficient reason for them to watch more action movies, all on its own.
Enjoying action movies is less important than preventing someone from being tortured, and if someone talks too much about trivial sources of fun in the context of immense suffering, then it makes sense to worry that they’re a bad person (or not sufficiently in touch with their compassion).
But I understand your position to be not “torture matters more than action movies”, but “action movies would ideally have zero impact on our decision-making, except insofar as it bears on suffering”. I gather that from your perspective, this is just taking compassion to its logical conclusion; assigning some more value to saving horrifically suffering people than to enjoying a movie is compassionate, so assigning infinitely more value to the one than the other seems like it’s just dialing compassion up to 11.
One reason I find this uncompelling is that I don’t think the right way to do compassion is to ignore most of the things people care about. I think that helping people requires doing the hard work of figuring out everything they value, and helping them get all those things. That might reduce to “just help them suffer less” in nearly all real-world decisions nowadays, because there’s an awful lot of suffering today; but that’s a contingent strategy based on various organisms’ makeup and environment in 2019, not the final word on everything that’s worth doing in a life.
To reject NU, is there some value you want to maximize beyond self-compassion and its role for preventing suffering, at the risk of allowing extreme suffering? How will you tell this to someone undergoing extreme suffering?
I’ll tell them I care a great deal about suffering, but I don’t assign literally zero importance to everything else.
NU people I’ve talked to often worry about scenarios like torture vs. dust specks, and that if we don’t treat happiness as literally of zero value, then we might make the wrong tradeoff and cause immense harm.
The flip side is dilemmas like:
Suppose you have a chance to push a button that will annihilate all life in the universe forever. You know for a fact that if you don’t push it, then billions of people will experience billions upon billions of years of happy, fulfilling, suffering-free life, filled with richness, beauty, variety, and complexity; filled with the things that make life most worth living, and with relationships and life-projects that people find deeply meaningful and satisfying.
However, you also know for a fact that if you don’t push the button, you’ll experience a tiny, almost-unnoticeable itch on your left shoulder blade a few seconds later, which will be mildly unpleasant for a second or two before the Utopian Future begins. With this one exception, no suffering will ever again occur in the universe, regardless of whether you push the button. Do you push the button, because your momentary itch matters more than all of the potential life and happiness you’d be cutting out?
I find negative utilitarianism unappealing for roughly the same reason I’d find “we should only care about disgust” or “we should only care about the taste of bananas” unappealing. Or if you think suffering is much closer to a natural kind than disgust, then supply some other mental (or physical!) state that seems more natural-kind-ish to you.
“Only suffering ultimately matters” and “only the taste of bananas ultimately matters” share the virtue of simplicity, but they otherwise run into the same difficulty, which is just that they don’t exhaustively describe all the things I enjoy or want or prefer. I don’t think my rejection of bananatarianism has to be any more complicated than that.
Something I wrote last year in response to a tangentially related paper:
I personally care about things other than suffering. What are negative utilitarians saying about that?
Are they saying that they don’t care about things like friendship, good food, joy, catharsis, adventure, learning new things, falling in love, etc., except as mechanisms for avoiding suffering? Are they saying that I’m deluded about having preferences like those? Are they saying that I should try to change my preferences — and if so, why? Are they saying that my preferences are fine in my personal decision-making as an individual, but shouldn’t get any weight in an idealized negotiation about what humanity as a group should do (ignoring any weight my preferences get from non-NU views that might in fact warrant a place at the bargaining table for more foundational or practical reasons distinct from the NU ideal) — and if so, why?
[...] “It’s wrong to ever base any decision whatsoever on my (or anyone else’s) enjoyment of anything whatsoever in life, except insofar as that enjoyment has downstream effects on other things” is an incredibly, amazingly strong claim. And it’s important in this context that you’re actually making that incredibly strong claim: more mild “negative-leaning” utilitarianisms (which probably shouldn’t be associated with NU, given how stark the difference is) don’t have to deal with the version of the world destruction argument I think x-risk people tend to be concerned about, which is not ‘in some scenarios, careful weighing of the costs and benefits can justify killing lots of people’ but rather ‘any offsets or alternatives to building misaligned resource-hungry AGI (without suffering subsystems) get literally zero weight, if you’re sufficiently confident that that’s what you’re building; there’s no need to even consider them; they aren’t even a feather on the scale’. I just don’t see why the not-even-a-feather-on-the-scale view deserves any more attention or respect than, e.g., divine-command theory; in an argument between the “negative-leaning” utilitarian and the real negative utilitarian, I don’t think the NU gets any good hits in.
(Simplicity is a virtue, but not when it’s of the “I’m going to attempt to disregard every consideration in all of my actions going forward except the expected amount of deliciousness in the future” or “… except the expected amount of lying in the future” variety; so simplicity on its own doesn’t raise the view to the level of having non-negligible probability compared to negative-learning U.)
Idea: if someone hovers over the karma number, a tooltip shows number of voters plus who non-anonymously upvoted; and if you click the karma number, it gives you an option to make your vote non-anonymous (which results in a private notification, plus a public notification if it’s an upvote).
This seems better to me than giving the “<” or “>” more functionality, since those are already pretty interactive and complex; whereas the number itself isn’t really doing much.
I agree with this worry, though I have a vague feeling that LW is capturing and retaining less of the rationalist core than is ideal — (EDIT: for example,) I feel like I see LW posts linked/discussed on social media less than is ideal. Not for the purpose of bringing in new readers, but just for the purpose of serving as a common-knowledge hub for rationalists. That’s just a feeling, though, and might reflect the bubbles I’m in. E.g., maybe LW is more of a thing on various Discords, since I don’t use Discord much.
If we’re getting fewer comments than we’d expect and desire given the number of posts or page visits, then that might also suggest that something’s wrong with the incentives for commenting.
An opt-in way to give non-anonymous upvotes (either publicly visible, or visible to the upvoted poster, or both) feels to me like it would help with issues in this space, since it’s a very low-effort way to give much more contentful/meaningful feedback than an anonymous upvote (“ah, Wei Dai liked my post” is way more information and reinforcement than “ah, my post has 4 more karma”, while being a lot less effort than Wei Dai writing more comments). Also separating out “I like this / I want to see more stuff like this” votes from “I agree with this” votes (where I think “I agree with this” votes should only publicly display when they’re non-anonymous). I feel like this helps with making posting more rewarding, and also just makes the site as a whole feel more hedonic and less impersonal.
Yes :) I wasn’t thinking real leather, though maybe synthetic leather also has signaling problems..!
Yep, there are good reasons to go for a cheaper edition (e.g., people can buy dozens of copies to pass out without breaking bank) and also to go for a more expensive edition. It makes sense to have one version that’s very optimized for affordability (the current version, which is good-quality but roughly at cost), and a separate version that’s optimized for other criteria. My main uncertainty is about which features Less Wrong readers are likely to care the most about, and how much those features are worth to them.
Not at present. Some people requested that we release higher-quality versions, so that’s been on our radar, and I’d be interested to hear what kinds of variants people would and wouldn’t be interested in buying. (Full-color, leather-bound, hardcover, etc.)
Yep! It doesn’t try to include literally every term or reference someone might want to google, but it includes terms like a priori, bit, deontology, directed acyclic graph, Everett branch, normative, and orthogonality, in addition to more rationality-specific terms. The kinds of terms we leave out are ones like “IRC” where some people might need to google the term, but it’s not really important enough to warrant a glossary entry.
The new version is under CC BY-NC-SA 4.0.
Hmm, Chrome seems to have modified the URLs. Try https://intelligence.org/rationality-ai-zombies/ and http://gumroad.com/l/mapterritory (no HTTPS) instead.
The rationalitybook.com link is currently a redirect to the MIRI R:AZ book page while we wait for TrikeApps to finish setting up the proper book page. I figured it was better to release now rather than wait for the finished website and the HACYM ebook, since the print edition will take a few days to deliver and some people will probably want to buy these as holiday presents.
(Amazon currently says it can deliver copies to me by Dec. 18-19 in California.)
New edition of “Rationality: From AI to Zombies”
K, cool. :)
Yeah, “Life is good” doesn’t validly imply “Living forever is good”. There can obviously be offsetting costs; I think it’s good to point this out, so we don’t confuse “there’s a presumption of evidence for (transhumanist intervention blah)” with “there’s an ironclad argument against any possible offsetting risks/costs turning up in the future”.
Like Said, I took Eliezer to just be saying “there’s no currently obvious reason to think that the optimal healthy lifespan for most people is <200 (or <1000, etc.).” My read is that 2007-Eliezer is trying to explain why bioconservatives need to point to some concrete cost at all (rather than taking it for granted that sci-fi-ish outcomes are weird and alien and therefore bad), and not trying to systematically respond to every particular scenario one might come up with where the utilities do flip at a certain age.
The goal is to provide an intuition pump: “Wanting people to live radically longer, be radically smarter, be radically happier, etc. is totally mundane and doesn’t require any exotic assumptions or bizarre preferences.” Pretty similar to another Eliezer intuition pump:In addition to standard biases, I have personally observed what look like harmful modes of thinking specific to existential risks. The Spanish flu of 1918 killed 25-50 million people. World War II killed 60 million people. is the order of the largest catastrophes in humanity’s written history. Substantially larger numbers, such as 500 million deaths, and especially qualitatively different scenarios such as the extinction of the entire human species, seem to trigger a different mode of thinking—enter into a “separate magisterium.” People who would never dream of hurting a child hear of an existential risk, and say, “Well, maybe the human species doesn’t really deserve to survive.”
There is a saying in heuristics and biases that people do not evaluate events, but descriptions of events—what is called non-extensional reasoning. The extension of humanity’s extinction includes the death of yourself, of your friends, of your family, of your loved ones, of your city, of your country, of your political fellows. Yet people who would take great offense at a proposal to wipe the country of Britain from the map, to kill every member of the Democratic Party in the U.S., to turn the city of Paris to glass—who would feel still greater horror on hearing the doctor say that their child had cancer— these people will discuss the extinction of humanity with perfect calm. “Extinction of humanity,” as words on paper, appears in fictional novels, or is discussed in philosophy books—it belongs to a different context than the Spanish flu. We evaluate descriptions of events, not extensions of events. The cliché phrase end of the world invokes the magisterium of myth and dream, of prophecy and apocalypse, of novels and movies. The challenge of existential risks to rationality is that, the catastrophes being so huge, people snap into a different mode of thinking.
People tend to think about the long-term future in Far Mode, which makes near-mode good things like “watching a really good movie” or “helping a sick child” feel less cognitively available/relevant/salient. The point of Eliezer’s “transhumanist proof by induction” isn’t to establish that there can never be offsetting costs (or diminishing returns, etc.) to having more of a good thing. It’s just to remind us that small concrete near-mode good things don’t stop being good when we talk about far-mode topics. (Indeed, they’re often the dominant consideration, because they can end up adding up to so much value when we talk about large-scale things.)
The Sequences make it seem like the Many Worlds interpretation has solved this problem but that’s not true.
No, Eliezer talks about this at some length. See The Born Probabilities:
[...] But what does the integral over squared moduli have to do with anything? On a straight reading of the data, you would always find yourself in both blobs, every time. How can you find yourself in one blob with greater probability? What are the Born probabilities, probabilities of? Here’s the map—where’s the territory?
I don’t know. It’s an open problem. Try not to go funny in the head about it.
This problem is even worse than it looks, because the squared-modulus business is the only non-linear rule in all of quantum mechanics. Everything else—everything else—obeys the linear rule that the evolution of amplitude distribution A, plus the evolution of the amplitude distribution B, equals the evolution of the amplitude distribution A + B.
When you think about the weather in terms of clouds and flapping butterflies, it may not look linear on that higher level. But the amplitude distribution for weather (plus the rest of the universe) is linear on the only level that’s fundamentally real.
Does this mean that the squared-modulus business must require additional physics beyond the linear laws we know—that it’s necessarily futile to try to derive it on any higher level of organization?
But even this doesn’t follow. [...]
And Privileging the Hypothesis:
[...] But, said Scott, we might encounter future evidence in favor of single-world quantum mechanics, and many-worlds still has the open question of the Born probabilities.
This is indeed what I would call the fallacy of privileging the hypothesis. There must be a trillion better ways to answer the Born question without adding a collapse postulate that would be the only non-linear, non-unitary, discontinous, non-differentiable, non-CPT-symmetric, non-local in the configuration space, Liouville’s-Theorem-violating, privileged-space-of-simultaneity-possessing, faster-than-light-influencing, acausal, informally specified law in all of physics. Something that unphysical is not worth saying out loud or even thinking about as a possibility without a rather large weight of evidence—far more than the current grand total of zero.
But because of a historical accident, collapse postulates and single-world quantum mechanics are indeed on everyone’s lips and in everyone’s mind to be thought of, and so the open question of the Born probabilities is offered up (by Scott Aaronson no less!) as evidence that many-worlds can’t yet offer a complete picture of the world. Which is taken to mean that single-world quantum mechanics is still in the running somehow.
In the minds of human beings, if you can get them to think about this particular hypothesis rather than the trillion other possibilities that are no more complicated or unlikely, you really have done a huge chunk of the work of persuasion. Anything thought about is treated as “in the running,” and if other runners seem to fall behind in the race a little, it’s assumed that this runner is edging forward or even entering the lead.
[… O]ur uncertainty about where the Born statistics come from should be uncertainty within the space of quantum theories that are continuous, linear, unitary, slower-than-light, local, causal, naturalistic, et cetera—the usual character of physical law. Some of that uncertainty might slop outside the standard space onto theories that violate one of these standard characteristics. It’s indeed possible that we might have to think outside the box. But single-world theories violate all these characteristics, and there is no reason to privilege that hypothesis.
The main claims Eliezer is criticizing in the QM sequence are that (1) reifying QM’s complex amplitudes runs afoul of Ockham’s Razor, (2) objective collapse is a plausible explanation for the Born probabilities, (3) QM shows that reality is ineffable, and (4) QM shows that there’s no such thing as reality. I don’t know what question of fact you think the Quantum Bayesians and Eliezer disagree about, or what novel factual claim QB is making. (I assume we agree ‘physical formalisms can be useful tools’ and ‘we can use probability theory to think about strength of belief’ aren’t novel claims.)
New thing: https://www.openphilanthropy.org/blog/new-web-app-calibration-training