Given all of that, I’d expect covert or overt government projects to continue even if we get a treaty banning private research. (Which I do find quite possible; I expect scary demos to happen naturally).
Why is this special to Shutdown, vs Controlled Takeoff? (Here, I’m specifically comparing two plans that both route through “first, do a pretty difficult political action of get countries agreeing to centralize GPUs”). If you just expect people to defect from that, what’s the point?
The scenario I’m thinking of mostly is the overt version, which is controlled takeoff. A few governments pursue controlled takeoff projects. This is hopefully done in the open and relatively collaboratively across US and Chinese teams. I’d assume they’d recruit from existing teams, effectively consolidating labs under government control. I’d hope that the Western and Chinese teams would agree to share their results on capabilities as well as alignment, although of course they’d worry about defection on that, too.
If they did it covertly that would be defecting. I haven’t thought about that scenario as much.
This scenario doesn’t seem like it would require consolidating GPUs, just monitoring their usage to some degree. It seems like it would be a lot easier to not make that part of the treaty.
The post is specifically about “globally controlled takeoff”, in which multiple governments have agreed to locate their GPUs in locations that are easy for each other to inspect.
There’s a spectrum between “Literally all countries agree to consolidate and monitor compute”, “US/UK/China do it”, “US/UK/Europe agree to do it among themselves”, “US does it just for itself” and “individual orgs are just being idk a bit careful and a bit cooperative in an ad-hoc fashion.”
I call the latter end of the spectrum “ad hoc semi-controlled semi-slowed takeof” at the beginning of the post. If we get something somewhere in the middle, seems probably an improvement.
I thought I was addressing the premise of your post: the world is ready to do serious restrictions on AI research: do they do shutdown or controlled takeoff?
I guess maybe I’m missing what’s important about the physical consolidation vs other methods of inspection and enforcement.
I think my scenario conforms to all of the gears you mention. It could be seen as adding another gear: the incentives/psychologies of government decision-makers.
Why is this special to Shutdown, vs Controlled Takeoff? (Here, I’m specifically comparing two plans that both route through “first, do a pretty difficult political action of get countries agreeing to centralize GPUs”). If you just expect people to defect from that, what’s the point?
The scenario I’m thinking of mostly is the overt version, which is controlled takeoff. A few governments pursue controlled takeoff projects. This is hopefully done in the open and relatively collaboratively across US and Chinese teams. I’d assume they’d recruit from existing teams, effectively consolidating labs under government control. I’d hope that the Western and Chinese teams would agree to share their results on capabilities as well as alignment, although of course they’d worry about defection on that, too.
If they did it covertly that would be defecting. I haven’t thought about that scenario as much.
This scenario doesn’t seem like it would require consolidating GPUs, just monitoring their usage to some degree. It seems like it would be a lot easier to not make that part of the treaty.
The post is specifically about “globally controlled takeoff”, in which multiple governments have agreed to locate their GPUs in locations that are easy for each other to inspect.
There’s a spectrum between “Literally all countries agree to consolidate and monitor compute”, “US/UK/China do it”, “US/UK/Europe agree to do it among themselves”, “US does it just for itself” and “individual orgs are just being idk a bit careful and a bit cooperative in an ad-hoc fashion.”
I call the latter end of the spectrum “ad hoc semi-controlled semi-slowed takeof” at the beginning of the post. If we get something somewhere in the middle, seems probably an improvement.
I thought I was addressing the premise of your post: the world is ready to do serious restrictions on AI research: do they do shutdown or controlled takeoff?
I guess maybe I’m missing what’s important about the physical consolidation vs other methods of inspection and enforcement.
I think my scenario conforms to all of the gears you mention. It could be seen as adding another gear: the incentives/psychologies of government decision-makers.