(Arguably too low. Even if MIRI crashes and somebody else carries on, I’d estimate a pretty high probability that their causal pathway there will have had something to do with MIRI. It is difficult to overstate just how much this problem was not on the horizon, at all, of work anyone could actually go out and do before MIRI.)
It doesn’t merely have to have something to do with MIRI, it must be the case that without funding MIRI we all die, and with funding MIRI, we don’t, and this is precisely the sort of thing that should have very low probability if MIRI is not demonstrably impressive at doing something else.
I do not see any slide showing a probability of 1 in 2000. Was this spoken aloud? At what time in the episode?
Hmm. It is mentioned here and other commenters there likewise talk of low probabilities. I guess I just couldn’t quite imagine someone seriously putting a non small probability on “with MIRI we live, without we die” aspect of it. Startups have quite small probability of success, even without attempting to do the impossible.
edit: And of course what actually matters is donor’s probability.
It doesn’t merely have to have something to do with MIRI, it must be the case that without funding MIRI we all die, and with funding MIRI, we don’t, and this is precisely the sort of thing that should have very low probability if MIRI is not demonstrably impressive at doing something else.
Hmm. It is mentioned here and other commenters there likewise talk of low probabilities. I guess I just couldn’t quite imagine someone seriously putting a non small probability on “with MIRI we live, without we die” aspect of it. Startups have quite small probability of success, even without attempting to do the impossible.
edit: And of course what actually matters is donor’s probability.