I have a feeling that the fundamental difference between your position and GiveWell’s arises not from a difference of opinion regarding mathematical arguments but because of a difference of values. Utilitarianism doesn’t say that I have to value potential people at anything approaching the level of value I assign to living persons. In particular, valuing potential persons at 0 negates many arguments that rely on speculative numbers to pump expected utility into the present, and I’m not even sure if it’s not right. Suppose that you had to choose between killing everyone currently alive at the end of their natural life spans, or murdering all but two people whom you were assured would repopulate the planet. My preference would be the former, despite it meaning the end of humanity. Valuing potential people without an extremely high discount rate also leads one to be strongly pro-life, to be against birth control programs in developing nations, etc.
Another possibility is that GiveWell’s true reason is based on the fact that recommending MIRI as an efficient charity would decrease their probability of becoming substantially larger (through attracting large numbers of mainstream donors). After they have more established credibility they would be able to direct a larger amount of money to existential charities, and recommending it now when it would reduce their growth trajectory could lower their impact in a fairly straightforward way unless the existential risk is truly imminent. But if they actually explicitly made this argument, it would undermine it’s whole point as they would be revealing their fringe intentions. Note that I actually think this would be a reasonable thing to do and am not trying to cast any aspersions on GiveWell.
I have a feeling that the fundamental difference between your position and GiveWell’s arises not from a difference of opinion regarding mathematical arguments but because of a difference of values.
Karnofsky has, as far as I know, not endorsed measures of charitable effectiveness that discount the utility of potential people. (On the other hand, as Nick Beckstead points out in a different comment and as is perhaps under-emphasized in the current version of the main post, neither has Karnofsky made a general claim that Bayesian adjustment defeats existential risk charity. He has only explicitly come out against “if there’s even a chance” arguments. But I think that in the context of his posts being reposted here on LW, many are likely to have interpreted them as providing a general argument that way, and I think it’s likely that the reasoning in the posts has at least something to do with why Karnofsky treats the category of existential risk charity as merely promising rather than as a main focus. For MIRI in particular, Karnofsky has specific criticisms that aren’t really related to the points here.)
In particular, valuing potential persons at 0 negates many arguments that rely on speculative numbers to pump expected utility into the present, and I’m not even sure if it’s not right.
While valuing potential persons at 0 makes existential risk versus other charities a closer call than if you included astronomical waste, I think the case is still fairly strong that the best existential risk charities save more expected currently-existing lives than the best other charities. The estimate from Anna Salamon’s talk linked in the main post makes investment into AI risk research roughly 4 orders of magnitude better for preventing the deaths of currently existing people than international aid charities. At the risk of anchoring, my guess is that the estimate is likely to be an overestimate, but not by 4 orders of magnitude. On the other hand, there may be non-existential risk charities that achieve greater returns in present lives but that also have factors barring them from being recommended by GiveWell.
Karnofsky has, as far as I know, not endorsed measures of charitable effectiveness that discount the utility of potential people.
Actually, according to this transcript on page four, Holden finds that the claim that the value of creating a life is “some reasonable” ratio of the value of saving a current life is very questionable. More exactly, the transcript sad:
Holden: So there is this hypothesis that the far future is worth n lives and
this causing this far future to exist is as good as saving n lives.
That I meant to state as an accurate characterization of someone
else’s view.
Eliezer: So I was about to say that it’s not my view that causing a life to
exist is on equal value of saving the life.
Holden: But it’s some reasonable multiplier.
Eliezer: But it’s some reasonable multiplier, yes. It’s not an order of
magnitude worse.
Holden: Right. I’m happy to modify it that way, and still say that I think
this is a very questionable hypothesis, but that I’m willing to
accept it for the sake of argument for a little bit. So yeah, then
my rejoinder, as like a parenthetical, which is not meant to pass
any Ideological Turing Test, it’s just me saying what I think, is
that this is very speculative, that it’s guessing at the number of
lives we’re going to have, and it’s also very debatable that you
should even be using the framework of applying a multiplier to
lives allowed versus lives saved. So I don’t know that that’s the
most productive discussion, it’s a philosophy discussion, often
philosophy discussions are not the most productive discussions
in my view.
Utilitarianism doesn’t say that I have to value potential people at anything approaching the level of value I assign to living persons.
In ten years time, you see a nine year old child fall into a pond. Do you save her from drowning? If so, you, in 2023, place value on people who aren’t born in 2013. If you don’t value those people now, in 2013, you’re temporally inconsistent.
Obviously this isn’t utilitarianism, but I think many people are unaware of this argument, despite its being from very common intuitions.
Valuing potential people without an extremely high discount rate also leads one to be strongly pro-life, to be against birth control programs in developing nations, etc.
Are these programs’ net desirability so self-evident that they constitute evidence against caring about future people? Yes, you could say “but they’re good for economic growth and the autonomy of women etc.”, those are reasons that would support supporting the programs even if we cared about future people. I think in general the desirability of contraception should be an output, rather than an input, to our expected value calculations.
On the other hand, if you’re the sort of person who doesn’t care about people far away in time, it might be sensible not to care about people far away in space.
In ten years time, you see a nine year old child fall into a pond. Do you save her from drowning? If so, you, in 2023, place value on people who aren’t born in 2013. If you don’t value those people now, in 2013, you’re temporally inconsistent.
What do you mean by “place value on people”? Your example is explained by placing value on the non-occurrence (or lateness) of their death. This is quite independent from placing value on the existence of people, and is therefore irrelevant to contraception, the continuation of humanity, etc.
You care about the deaths of people without caring about people?
What if I changed the example—and it’s about whether or not to help educate the child, or comport her, or feed her. Do we are about the education, hunger and happiness of the child also, without caring about the child?
You can say that a death averted or delayed is a good thing without being committed to saying that a birth is a good thing. That’s the point I was trying to make.
Similarly, you can “care about people” in the sense that you think that, given that a person exists, they should have a good life, without thinking that a world with people who have good lives is better than a world with no people at all.
No you can’t. Consider three worlds, only differing with regards person A.
In world 1, U(A) = 20.
In world 2, U(A) = 10.
In world 3, U(A) = undefined, as A does not exist.
Which world is best? As we agree that people who exist should have a good life, U(1) > U(2). Assume U(2)=U(3), as per your suggest that we’re unconcerned about people’s existence/non-existence. Therefore, by transitivity of preference, U(1) > U(3). So we do care about A’s existence or non-existence.
But U(3) = U(2) doesn’t reflect what I was suggesting. There’s nothing wrong with assuming U(3) ≥ U(1). You can care about A even though you think that it would have been better if they hadn’t been born. You’re right, though, about the conclusion that it’s difficult to be unconcerned with a person’s existence. Cases of true indifference about a person’s birth will be rare.
Personally, I can imagine a world with arbitrarily happy people and it doesn’t feel better to me than a world where those people are never been born; and this doesn’t feel inconsistent. And as long as the utility I can derive from people’s happiness is bounded, it isn’t.
Wonderful post. Thank you.
I have a feeling that the fundamental difference between your position and GiveWell’s arises not from a difference of opinion regarding mathematical arguments but because of a difference of values. Utilitarianism doesn’t say that I have to value potential people at anything approaching the level of value I assign to living persons. In particular, valuing potential persons at 0 negates many arguments that rely on speculative numbers to pump expected utility into the present, and I’m not even sure if it’s not right. Suppose that you had to choose between killing everyone currently alive at the end of their natural life spans, or murdering all but two people whom you were assured would repopulate the planet. My preference would be the former, despite it meaning the end of humanity. Valuing potential people without an extremely high discount rate also leads one to be strongly pro-life, to be against birth control programs in developing nations, etc.
Another possibility is that GiveWell’s true reason is based on the fact that recommending MIRI as an efficient charity would decrease their probability of becoming substantially larger (through attracting large numbers of mainstream donors). After they have more established credibility they would be able to direct a larger amount of money to existential charities, and recommending it now when it would reduce their growth trajectory could lower their impact in a fairly straightforward way unless the existential risk is truly imminent. But if they actually explicitly made this argument, it would undermine it’s whole point as they would be revealing their fringe intentions. Note that I actually think this would be a reasonable thing to do and am not trying to cast any aspersions on GiveWell.
Karnofsky has, as far as I know, not endorsed measures of charitable effectiveness that discount the utility of potential people. (On the other hand, as Nick Beckstead points out in a different comment and as is perhaps under-emphasized in the current version of the main post, neither has Karnofsky made a general claim that Bayesian adjustment defeats existential risk charity. He has only explicitly come out against “if there’s even a chance” arguments. But I think that in the context of his posts being reposted here on LW, many are likely to have interpreted them as providing a general argument that way, and I think it’s likely that the reasoning in the posts has at least something to do with why Karnofsky treats the category of existential risk charity as merely promising rather than as a main focus. For MIRI in particular, Karnofsky has specific criticisms that aren’t really related to the points here.)
While valuing potential persons at 0 makes existential risk versus other charities a closer call than if you included astronomical waste, I think the case is still fairly strong that the best existential risk charities save more expected currently-existing lives than the best other charities. The estimate from Anna Salamon’s talk linked in the main post makes investment into AI risk research roughly 4 orders of magnitude better for preventing the deaths of currently existing people than international aid charities. At the risk of anchoring, my guess is that the estimate is likely to be an overestimate, but not by 4 orders of magnitude. On the other hand, there may be non-existential risk charities that achieve greater returns in present lives but that also have factors barring them from being recommended by GiveWell.
Actually, according to this transcript on page four, Holden finds that the claim that the value of creating a life is “some reasonable” ratio of the value of saving a current life is very questionable. More exactly, the transcript sad:
I agree; this is excelent.
In ten years time, you see a nine year old child fall into a pond. Do you save her from drowning? If so, you, in 2023, place value on people who aren’t born in 2013. If you don’t value those people now, in 2013, you’re temporally inconsistent.
Obviously this isn’t utilitarianism, but I think many people are unaware of this argument, despite its being from very common intuitions.
Are these programs’ net desirability so self-evident that they constitute evidence against caring about future people? Yes, you could say “but they’re good for economic growth and the autonomy of women etc.”, those are reasons that would support supporting the programs even if we cared about future people. I think in general the desirability of contraception should be an output, rather than an input, to our expected value calculations.
On the other hand, if you’re the sort of person who doesn’t care about people far away in time, it might be sensible not to care about people far away in space.
What do you mean by “place value on people”? Your example is explained by placing value on the non-occurrence (or lateness) of their death. This is quite independent from placing value on the existence of people, and is therefore irrelevant to contraception, the continuation of humanity, etc.
You care about the deaths of people without caring about people?
What if I changed the example—and it’s about whether or not to help educate the child, or comport her, or feed her. Do we are about the education, hunger and happiness of the child also, without caring about the child?
You can say that a death averted or delayed is a good thing without being committed to saying that a birth is a good thing. That’s the point I was trying to make.
Similarly, you can “care about people” in the sense that you think that, given that a person exists, they should have a good life, without thinking that a world with people who have good lives is better than a world with no people at all.
No you can’t. Consider three worlds, only differing with regards person A.
In world 1, U(A) = 20.
In world 2, U(A) = 10.
In world 3, U(A) = undefined, as A does not exist.
Which world is best? As we agree that people who exist should have a good life, U(1) > U(2). Assume U(2)=U(3), as per your suggest that we’re unconcerned about people’s existence/non-existence. Therefore, by transitivity of preference, U(1) > U(3). So we do care about A’s existence or non-existence.
But U(3) = U(2) doesn’t reflect what I was suggesting. There’s nothing wrong with assuming U(3) ≥ U(1). You can care about A even though you think that it would have been better if they hadn’t been born. You’re right, though, about the conclusion that it’s difficult to be unconcerned with a person’s existence. Cases of true indifference about a person’s birth will be rare.
Personally, I can imagine a world with arbitrarily happy people and it doesn’t feel better to me than a world where those people are never been born; and this doesn’t feel inconsistent. And as long as the utility I can derive from people’s happiness is bounded, it isn’t.
U(2)=U(3) isn’t “a world with people who have good lives is not better than a world with no people at all”. That would be U(1)=U(3).