I never said this was a good plan? I agree with the post’s overall argument that we should work on a pause ASAP. I just don’t think this particular argument—that it takes time to implement a proper pause—is very strong, since it wouldn’t take long to implement something basic and work out the details later.
The “there will be no warning shots” claim was always the security mindset talking: it could equally be phrased “we should be able to survive even if there are no warning shots” and “a rogue AI smarter than you has a strong incentive not to give you a warning shot but instead bide its time until it’s sure of sucess (unless that means you might meanwhile build something else smarter than it)”.
IMO, we currently have a fusillade of minor warning shots going on, and as people rush to add agents to the economy, we’ll get louder ones.
I never said this was a good plan? I agree with the post’s overall argument that we should work on a pause ASAP. I just don’t think this particular argument—that it takes time to implement a proper pause—is very strong, since it wouldn’t take long to implement something basic and work out the details later.
I think a pause would take a long(ish) time in the mainline scenario:
Develop enough support for a pause among policy-makers and the public.
Write proposals for how to implement a pause; figure out how to implement the details (how to do GPU tracking etc.).
Set up negotiations between relevant countries.
Set up more negotiations when the first round in all likelihood goes nowhere.
The governments agree to implement a pause starting 2 years from the negotiations, because that’s how these things usually work.
We’re already several years deep into steps 1 and 2 and there’s still a long way to go.
The “there will be no warning shots” claim was always the security mindset talking: it could equally be phrased “we should be able to survive even if there are no warning shots” and “a rogue AI smarter than you has a strong incentive not to give you a warning shot but instead bide its time until it’s sure of sucess (unless that means you might meanwhile build something else smarter than it)”.
IMO, we currently have a fusillade of minor warning shots going on, and as people rush to add agents to the economy, we’ll get louder ones.