I agree, though I think the risk of a secret ASI project being successful with limited resources probably increases significantly the longer the timeline, based on a basic fragile-world extrapolation. The group of actors with capability to reach ASI will expand from major powers to middle powers and eventually small nations (some of which may be nuclear pariah states) and private groups. All it takes is one group to pull off a successful secret project (or possibly a not-so-secret one if it’s run by a country with enough nukes) to break the equilibrium.
Assuming the above, then an AI pause is doable, but not an indefinite ASI ban. Eventually the risk of secret projects is large enough that the world as a whole needs to resume AI development to stay ahead of one if it existed.
FWIW, I think this is a problem for decades in the future. I agree that an ASI ban doesn’t solve the problem indefinitely, but we’ll have extra decades to figure out what to do.
I agree, though I think the risk of a secret ASI project being successful with limited resources probably increases significantly the longer the timeline, based on a basic fragile-world extrapolation. The group of actors with capability to reach ASI will expand from major powers to middle powers and eventually small nations (some of which may be nuclear pariah states) and private groups. All it takes is one group to pull off a successful secret project (or possibly a not-so-secret one if it’s run by a country with enough nukes) to break the equilibrium.
Assuming the above, then an AI pause is doable, but not an indefinite ASI ban. Eventually the risk of secret projects is large enough that the world as a whole needs to resume AI development to stay ahead of one if it existed.
FWIW, I think this is a problem for decades in the future. I agree that an ASI ban doesn’t solve the problem indefinitely, but we’ll have extra decades to figure out what to do.