if you’re worried that a nuclear weapon with yield Y might ignite the atmosphere, it doesn’t help to first test a nuclear weapon with yield 0.1×Y, and then if the atmosphere hasn’t been ignited yet, next try testing one with yield 0.2×Y, etc.” I.e., I sometimes (not always!) see people talking about gradual AI progress without having a clear and plausible (to me) mechanism by which the earlier steps actually de-risk the later steps.
In this example, people believe that each subsequent nuclear test could solve the problem of atmospheric ignite in the future.
In this example, people believe that each subsequent nuclear test could solve the problem of atmospheric ignite in the future.