That will tell you that you shouldn’t expect to spend more to save one life than estimated from the top charity; it might even tell you that the expected cost of the top rated charity is higher than the estimated cost from the next-ranked charity.
Without a reason to believe that the accuracy of the estimate for the lower-ranked charity is greater, you can’t conclude that the expected cost of the higher-ranked charity is greater than the expected cost of the lower-ranked charity.
I don’t see any need to “solve” the “problem”, at least in this context. The goal is maximize utility, not to minimize “surprise”. All we care about is the ordinal values of the choices; getting the quantitative values right is simply a means to that end. The “solution” presented in your article doesn’t make any sense; unless there’s some asymmetry between the choices, how can your “solution” change the ordinal value of the choices?
Yes, but the exact values for dollars per life saved given by GiveWell are both relevant in the article we’re discussing and is also frequently cited here on LessWrong.
This is called the optimizer’s curse. There are standard ways to solve the problem, e.g. multilevel modeling.
Good to know, thanks!
That will tell you that you shouldn’t expect to spend more to save one life than estimated from the top charity; it might even tell you that the expected cost of the top rated charity is higher than the estimated cost from the next-ranked charity.
Without a reason to believe that the accuracy of the estimate for the lower-ranked charity is greater, you can’t conclude that the expected cost of the higher-ranked charity is greater than the expected cost of the lower-ranked charity.
I don’t see any need to “solve” the “problem”, at least in this context. The goal is maximize utility, not to minimize “surprise”. All we care about is the ordinal values of the choices; getting the quantitative values right is simply a means to that end. The “solution” presented in your article doesn’t make any sense; unless there’s some asymmetry between the choices, how can your “solution” change the ordinal value of the choices?
Yes, but the exact values for dollars per life saved given by GiveWell are both relevant in the article we’re discussing and is also frequently cited here on LessWrong.