I see. I thought you were more in tune with Eliezer on this issue. I was simply trying to see what would make me take the MIRI research much more seriously. I am fascinated by the mathematical side of it, which is hopefully of high enough quality to attract expert attention, but I am currently much more skeptical of its effects on the odds of humanity surviving the next century or two.
I changed specifics to variables because I was interested more in the broader point than the specific case.
Asteroid tracking involved spending ~$100MM to eliminate most of the expected losses from civilization-wrecking asteroids. Generously, it might have eliminated as much as a 10^-6 extinction risk (if we had found a dinosaur-killer on course our civilization would have mobilized to divert it). At the same tradeoff, getting rid of a 9% extinction risk would seem to be worth $9T or more. Billions are spent on biodefense and nuclear nonproliferation programs each year.
So it seems to me that a 9% figure ‘overshoots’ the relevant thresholds in other areas: a much lower believed cost per increment of existential risk reduction would seem to suffice for more-than-adequate support (e.g. national governments, large foundations, and plenty of scientific talent would step in before that, based on experiences with nuclear weapons, climate change, cancer research, etc).
For comparison, consider someone who says that she will donate to malaria relief iff there is solidly convincing proof that at least 1000 cases of malaria affecting current people will be averted per dollar in the short-term. This is irrelevant in a world with a Gates Foundation, GiveWell, and so on: she will never get the chance as those with less stringent thresholds act.
I was trying to clarify whether you were using an extreme example to make the point in principle, or were saying that your threshold for action would actually be in that vicinity.
I see. I thought you were more in tune with Eliezer on this issue. I was simply trying to see what would make me take the MIRI research much more seriously. I am fascinated by the mathematical side of it, which is hopefully of high enough quality to attract expert attention, but I am currently much more skeptical of its effects on the odds of humanity surviving the next century or two.
I changed specifics to variables because I was interested more in the broader point than the specific case.
Asteroid tracking involved spending ~$100MM to eliminate most of the expected losses from civilization-wrecking asteroids. Generously, it might have eliminated as much as a 10^-6 extinction risk (if we had found a dinosaur-killer on course our civilization would have mobilized to divert it). At the same tradeoff, getting rid of a 9% extinction risk would seem to be worth $9T or more. Billions are spent on biodefense and nuclear nonproliferation programs each year.
So it seems to me that a 9% figure ‘overshoots’ the relevant thresholds in other areas: a much lower believed cost per increment of existential risk reduction would seem to suffice for more-than-adequate support (e.g. national governments, large foundations, and plenty of scientific talent would step in before that, based on experiences with nuclear weapons, climate change, cancer research, etc).
For comparison, consider someone who says that she will donate to malaria relief iff there is solidly convincing proof that at least 1000 cases of malaria affecting current people will be averted per dollar in the short-term. This is irrelevant in a world with a Gates Foundation, GiveWell, and so on: she will never get the chance as those with less stringent thresholds act.
I was trying to clarify whether you were using an extreme example to make the point in principle, or were saying that your threshold for action would actually be in that vicinity.