97.3% chance of having no effect (all parameters are changeable by the way)
70% chance of positive effect conditional on the above not occurring, and hence
30% chance of negative effect, which leads to
30% increase in probability of extinction (relative to the positive counterfactual’s effect, not total p(doom))
The exact figures RP’s CCM spits out aren’t that meaningful; what’s more interesting for me are the estimates under alternative weighting schemes for incorporating risk aversion (Table 1), pretty much all of which are negative. My main takeaway from this sort of exercise is the importance of reducing sign uncertainty, which isn’t a new point but sometimes appears underemphasized.
As an aside, Rethink Priorities’ cross-cause cost-effectiveness (CCM) model automatically prompts consideration of downside risk as part of the calculation template so to speak. Their placeholder values for a generic AI misalignment x-risk mitigation megaproject are
97.3% chance of having no effect (all parameters are changeable by the way)
70% chance of positive effect conditional on the above not occurring, and hence
30% chance of negative effect, which leads to
30% increase in probability of extinction (relative to the positive counterfactual’s effect, not total p(doom))
The exact figures RP’s CCM spits out aren’t that meaningful; what’s more interesting for me are the estimates under alternative weighting schemes for incorporating risk aversion (Table 1), pretty much all of which are negative. My main takeaway from this sort of exercise is the importance of reducing sign uncertainty, which isn’t a new point but sometimes appears underemphasized.