Thanks for the thoughtful comments. Without responding to all threads, I’d like to address a few of the themes that came up. FYI, there are also interesting discussions of this post at The GiveWell Blog , Overcoming Bias , and Quomodocumque (the latter includes Terence Tao’s thoughts on “Pascal’s Mugging”).
On what I’m arguing. There seems to be confusion on which of the following I am arguing:
(1) The conceptual idea of maximizing expected value is problematic.
(2) Explicit estimates of expected value are problematic and can’t be taken literally.
(3) Explicit estimates of expected value are problematic/can’t be taken literally when they don’t include a Bayesian adjustment of the kind outlined in my post.
As several have noted, I do not argue (1). I do aim to give with the aim of maximizing expected good accomplished, and in particular I consider myself risk-neutral in giving.
I strongly endorse (3) and there doesn’t seem to be disagreement on this point.
I endorse (2) as well, though less strongly than I endorse (3). I am open to the idea of formally performing a Bayesian adjustment, and if this formalization is well done enough, taking the adjusted expected-value estimate literally. However,
I have examined a lot of expected-value estimates relevant to giving, including those done by the DCP2 , Copenhagen Consensus , and Poverty Action Lab , and have never once seen a formalized adjustment of this kind.
If you disagree with the above point, and feel that such adjustments ought to be done formally, then you do disagree with a substantial part of my post; however, you ought to find the remainder of the post more consequential than I do, since it implies substantial room for improvement in the most prominent cost-effectiveness estimates (and perhaps all cost-effectiveness estimates) in the domains under discussion.
All of the above applies to expected-value calculations that take relatively large amounts of guesswork, such as in the domain of giving. There are expected-value estimates that I feel are precise/robust enough to take literally.
Is it reasonable to model existential risk reduction and/or “Pascal’s Mugging” using log-/normal distributions? Several have pointed out that existential risk reduction and “Pascal’s Mugging” seem to involve “either-or” scenarios that aren’t well approximated by log-/normal distributions. I wish to emphasize that I’m focused on the prior over expected value of one’s actions and on the distribution of error in one’s expected-value estimate. (The latter is a fuzzy concept that may be best formalized with the aid of concepts such as imprecise probability. In the scenarios under discussion, one often must estimate the probability of catastrophe essentially by making a wild guess with a wide confidence interval, leaving wide room for “estimate error” around the expected-value calculation.) Bayesian adjustments to expected-value estimates of actions, in this framework, are smaller (all else equal) for well-modeled and well-understood “either-or” scenarios than for poorly-modeled and poorly-understood “either-or” scenarios.
For both the prior and for the “estimate error,” I think the log-/normal distribution can be a reasonable approximation, especially when considering the uncertainty around the impact of one’s actions on the probability of catastrophe.
The basic framework of this post still applies, and many of its conclusions may as well, even when other types of probability distributions are assumed.
My views on existential risk reduction are outside the scope of this post. The only mention I make of existential risk reduction is to critique the argument that “charities working on reducing the risk of sudden human extinction must be the best ones to support, since the value of saving the human race is so high that ‘any imaginable probability of success’ would lead to a higher expected value for these charities than for others.” Note that Eliezer Yudkowsky and Michael Vassar also appear to disapprove of this argument, so it seems clear that disputing this argument is not the same as arguing against existential risk reduction charities.
For the past few years we have considered catastrophic risk reduction charities to be lower on GiveWell’s priority list for investigation than developing-world aid charities, but still relatively high on the list in the scheme of things. I’ve recently started investigating these causes a bit more, starting with SIAI (see LW posts on my discussion with SIAI representatives and my exchange with Jaan Tallinn). It’s plausible to me that asteroid risk reduction is a promising area, but I haven’t looked into it enough (yet) to comment more on that.
My informal objections to what I term EEV. Several have criticized the section of my post giving informal objections to what I term the EEV approach (by which I meant explicitly estimating expected value using a rough calculation and not performing a Bayesian adjustment). This section was intended only as a very rough sketch of what unnerves me about EEV; there doesn’t seem to be much dispute over the more formal argument I made against EEV; thus, I don’t plan on responding to critiques of this section.
With respect to charitable giving, do you have any advise for people who lack the math education to do their own calculations or to survey the available evaluations? Should one decide between Karnofsky and Yudkowsky solely based on one’s intuition or postpone the decision and concentrate on acquiring the necessary background knowledge?
Why not use several different methodologies on GiveWell, instead of just one, since there is some disagreement over methodologies? I can understand giving your favorite methodology top billing, of course (both because you believe it is best and it is your site and also to avoid confusion among donors), but there seems to be room for more than one.
Hello all,
Thanks for the thoughtful comments. Without responding to all threads, I’d like to address a few of the themes that came up. FYI, there are also interesting discussions of this post at The GiveWell Blog , Overcoming Bias , and Quomodocumque (the latter includes Terence Tao’s thoughts on “Pascal’s Mugging”).
On what I’m arguing. There seems to be confusion on which of the following I am arguing:
(1) The conceptual idea of maximizing expected value is problematic.
(2) Explicit estimates of expected value are problematic and can’t be taken literally.
(3) Explicit estimates of expected value are problematic/can’t be taken literally when they don’t include a Bayesian adjustment of the kind outlined in my post.
As several have noted, I do not argue (1). I do aim to give with the aim of maximizing expected good accomplished, and in particular I consider myself risk-neutral in giving.
I strongly endorse (3) and there doesn’t seem to be disagreement on this point.
I endorse (2) as well, though less strongly than I endorse (3). I am open to the idea of formally performing a Bayesian adjustment, and if this formalization is well done enough, taking the adjusted expected-value estimate literally. However,
I have examined a lot of expected-value estimates relevant to giving, including those done by the DCP2 , Copenhagen Consensus , and Poverty Action Lab , and have never once seen a formalized adjustment of this kind.
I believe that often—particularly in the domains discussed here—formalizing such an adjustment in a reasonable way is simply not feasible and that using intuition is superior. This is argued briefly in this post, and Dario Amodei and Jonah Sinick have an excellent exchange further exploring this idea at the GiveWell Blog.
If you disagree with the above point, and feel that such adjustments ought to be done formally, then you do disagree with a substantial part of my post; however, you ought to find the remainder of the post more consequential than I do, since it implies substantial room for improvement in the most prominent cost-effectiveness estimates (and perhaps all cost-effectiveness estimates) in the domains under discussion.
All of the above applies to expected-value calculations that take relatively large amounts of guesswork, such as in the domain of giving. There are expected-value estimates that I feel are precise/robust enough to take literally.
Is it reasonable to model existential risk reduction and/or “Pascal’s Mugging” using log-/normal distributions? Several have pointed out that existential risk reduction and “Pascal’s Mugging” seem to involve “either-or” scenarios that aren’t well approximated by log-/normal distributions. I wish to emphasize that I’m focused on the prior over expected value of one’s actions and on the distribution of error in one’s expected-value estimate. (The latter is a fuzzy concept that may be best formalized with the aid of concepts such as imprecise probability. In the scenarios under discussion, one often must estimate the probability of catastrophe essentially by making a wild guess with a wide confidence interval, leaving wide room for “estimate error” around the expected-value calculation.) Bayesian adjustments to expected-value estimates of actions, in this framework, are smaller (all else equal) for well-modeled and well-understood “either-or” scenarios than for poorly-modeled and poorly-understood “either-or” scenarios.
For both the prior and for the “estimate error,” I think the log-/normal distribution can be a reasonable approximation, especially when considering the uncertainty around the impact of one’s actions on the probability of catastrophe.
The basic framework of this post still applies, and many of its conclusions may as well, even when other types of probability distributions are assumed.
My views on existential risk reduction are outside the scope of this post. The only mention I make of existential risk reduction is to critique the argument that “charities working on reducing the risk of sudden human extinction must be the best ones to support, since the value of saving the human race is so high that ‘any imaginable probability of success’ would lead to a higher expected value for these charities than for others.” Note that Eliezer Yudkowsky and Michael Vassar also appear to disapprove of this argument, so it seems clear that disputing this argument is not the same as arguing against existential risk reduction charities.
For the past few years we have considered catastrophic risk reduction charities to be lower on GiveWell’s priority list for investigation than developing-world aid charities, but still relatively high on the list in the scheme of things. I’ve recently started investigating these causes a bit more, starting with SIAI (see LW posts on my discussion with SIAI representatives and my exchange with Jaan Tallinn). It’s plausible to me that asteroid risk reduction is a promising area, but I haven’t looked into it enough (yet) to comment more on that.
My informal objections to what I term EEV. Several have criticized the section of my post giving informal objections to what I term the EEV approach (by which I meant explicitly estimating expected value using a rough calculation and not performing a Bayesian adjustment). This section was intended only as a very rough sketch of what unnerves me about EEV; there doesn’t seem to be much dispute over the more formal argument I made against EEV; thus, I don’t plan on responding to critiques of this section.
With respect to charitable giving, do you have any advise for people who lack the math education to do their own calculations or to survey the available evaluations? Should one decide between Karnofsky and Yudkowsky solely based on one’s intuition or postpone the decision and concentrate on acquiring the necessary background knowledge?
Why not use several different methodologies on GiveWell, instead of just one, since there is some disagreement over methodologies? I can understand giving your favorite methodology top billing, of course (both because you believe it is best and it is your site and also to avoid confusion among donors), but there seems to be room for more than one.