But it looks like the shape of the distributions isn’t normal-like? In fact, that’s one of the standard EA arguments for why it’s important to spend energy on finding the most effective thing you can do: if possible intervention outcomes really were approximately normally distributed, then your exact choice of an intervention wouldn’t matter all that much. But actually the distribution of outcomes looks very skewed; to quote The moral imperative towards cost-effectiveness:
DCP2 includes cost-effectiveness
estimates for 108 health interventions, which are presented in the chart below,
arranged from least effective to most effective [...] This larger sample of interventions is even more disparate in terms of costeffectiveness.
The least effective intervention analysed is still the treatment for
Kaposi’s sarcoma, but there are also interventions up to ten times more cost-effective
than education for high risk groups. In total, the interventions are spread over more
than four orders of magnitude, ranging from 0.02 to 300 DALYs per $1,000, with a
median of 5. Thus, moving money from the least effective intervention to the most
effective would produce about 15,000 times the benefit, and even moving it from the
median intervention to the most effective would produce about 60 times the benefit.
It can also be seen that due to the skewed distribution, the most effective
interventions produce a disproportionate amount of the benefits. According to the
DCP2 data, if we funded all of these interventions equally, 80% of the benefits would
be produced by the top 20% of the interventions. [...]
Moreover, there have been health interventions that are even more effective than any
of those studied in the DCP2. [...] For instance in the case of
smallpox, the total cost of eradication was about $400 million. Since more than 100
million lives have been saved so far, this has come to less than $4 per life saved —
significantly superior to all interventions in the DCP2.
I think you misunderstood what I said or I didn’t explain myself well: I’m not assuming that the DALY distribution obtained if you choose interventions at random is normal. I’m assuming that for each intervention, the DALY distribution it produces is normal, with an intervention-dependent mean and variance.
I think that for the kind of interventions that GiveWell considers, this is a reasonable assumption: if the number of DALYs produced by each intervention is the result of a sum of many roughly independent variables (e.g. DALYs gained by helping Alice, DALYs gained by helping Bob, etc.) the total should be approximately normally distributed, due to the central limit theorem.
For other types of interventions, e.g. whether to fund a research project, you may want to use a more general family of distributions that allows non-zero skewness (e.g. skew-normal distributions), but as long as the distribution is light-tailed and you don’t use extreme values for the parameters, you would not run into Pascal’s Mugging issues.
But it looks like the shape of the distributions isn’t normal-like? In fact, that’s one of the standard EA arguments for why it’s important to spend energy on finding the most effective thing you can do: if possible intervention outcomes really were approximately normally distributed, then your exact choice of an intervention wouldn’t matter all that much. But actually the distribution of outcomes looks very skewed; to quote The moral imperative towards cost-effectiveness:
I think you misunderstood what I said or I didn’t explain myself well: I’m not assuming that the DALY distribution obtained if you choose interventions at random is normal. I’m assuming that for each intervention, the DALY distribution it produces is normal, with an intervention-dependent mean and variance.
I think that for the kind of interventions that GiveWell considers, this is a reasonable assumption: if the number of DALYs produced by each intervention is the result of a sum of many roughly independent variables (e.g. DALYs gained by helping Alice, DALYs gained by helping Bob, etc.) the total should be approximately normally distributed, due to the central limit theorem.
For other types of interventions, e.g. whether to fund a research project, you may want to use a more general family of distributions that allows non-zero skewness (e.g. skew-normal distributions), but as long as the distribution is light-tailed and you don’t use extreme values for the parameters, you would not run into Pascal’s Mugging issues.