I think I disagree. Based on your presentation here, I think someone following a policy inspired by this post would be more likely to cause existential catastrophe by pursuing a promising false positive that actually destroys all future value in our Hubble volume. I’ve argued we need to focus on minimizing false positive risk rather than optimizing for max expected value, which is what I read this as proposing we do.
I think I disagree. Based on your presentation here, I think someone following a policy inspired by this post would be more likely to cause existential catastrophe by pursuing a promising false positive that actually destroys all future value in our Hubble volume. I’ve argued we need to focus on minimizing false positive risk rather than optimizing for max expected value, which is what I read this as proposing we do.