But are you strongly claiming that it’s infeasible? If so, that’s the position I’d like to understand—why do you think that, if you do? Is this a case that’s been worked out and explained somewhere? Has it been debated seriously?
I don’t think you can talk about feasibility in a vacuum. Is it feasible that Congress pass a constitutional amendment removing the electoral college that is ratified by the states? From a technical sense, of course. From my perspective as someone who wants that to happen, not really.
Let me try to lay out my overall position.
I think right now you can break players into three categories
If all players are in group 1 (and knew the others were in group 1) a global pause is trivial. If all players are in groups 1 and 2 a global pause is a technical question: Can you create a treaty/system strong enough that secret defection is not possible? Players in group 3 have no interest in a pause and would have to be brought to the table with other incentives.
It’s very hard to know who is in which camp. I’d guess lab employees are largely split between 1, 2, and 3. I think the companies themselves are probably in groups 2 or 3.I think the U.S. government is in group 3 (caveat that there are lots of competing voices here and I have a very hard time modelling Trump). I have no idea where China or other world powers are.
The groups are more of a spectrum than binary. Someone in group 2 who thinks the status quo is very negative will be willing to spend more resources on a pause than someone who more loosely prefers a pause.
So I think the question of feasibility is how feasible is it for people in groups 1 and 2 to get the U.S. government into group 2 and strongly enough in group 2 that it is willing to expend lots of resources on a treaty (and I think since there are other players in group 3 you would need lots of resources). I don’t have a rock solid case that this is infeasible (I don’t think you ever really could), but I think it likely is at this time. EDIT: Adding in a little more here because I saw your conversations with others and think this may be our central disagreement. I think many people overrate how politically salient AI is. Anti-AI sentiment is all over the place, but I think its a mile wide and an inch deep. When you look at Gallup polling of the most important issues facing our country, AI doesn’t show up. I think any politicians that did serious damage to the U.S. economy and potentially started wars to pause AI would be electorally punished. I think if there are anti-AI legislation that is electorally valuable it will probably be surface level stuff that doesn’t move the needle on AI risk (e.g. data center water regulations). Now voters are not an immovable force, but I think it will be hard to move them quickly and may take some other shock to the system (impactful warning shot, widespread job loss, etc.)
Neither of these arguments make sense to me, and seem quite opposite the truth. Like, we should stop ASAP so that we’re not in a terrible time crunch, right?
As for why we might want to wait for a pause if AGI is further away, If we could have a strong pause now, I’d definitely agree with you. However, I think if you were to push the world into a weakly enforced pause it may be harder to get a strong pause down the line. In the status quo I hope we can slow down capabilities work, build up chip control systems and other monitoring options and as AI gets more powerful (hopefully before we all die) people will move towards a real pause. My concern is that a weak pause drives AI development underground, differentially hurts safety, and doesn’t allow people to update in the direction of a real pause. Like a think a world where AI development is nominally illegal, but the Chinese and U.S. Governments both had well funded secret programs is much worse than Evan’s proposal and likely worse than the status quo.
FWIW if I were a world dictator I would implement a global pause followed by heavily regulated and monitored technical alignment research. I also weakly hold the belief that people who think we are in an emergency situation should state that clearly and strongly, but have found some arguments for being more strategic compelling (e.g. Holden Karnofsky on the 80,000 hours podcast).
I don’t think you can talk about feasibility in a vacuum. Is it feasible that Congress pass a constitutional amendment removing the electoral college that is ratified by the states? From a technical sense, of course. From my perspective as someone who wants that to happen, not really.
Let me try to lay out my overall position.
I think right now you can break players into three categories
Those who model the situation as a stag hunt.
Those that model the situation as a prisoner’s dilemma.
Those that model the situation as a deadlock.
If all players are in group 1 (and knew the others were in group 1) a global pause is trivial. If all players are in groups 1 and 2 a global pause is a technical question: Can you create a treaty/system strong enough that secret defection is not possible? Players in group 3 have no interest in a pause and would have to be brought to the table with other incentives.
It’s very hard to know who is in which camp. I’d guess lab employees are largely split between 1, 2, and 3. I think the companies themselves are probably in groups 2 or 3.I think the U.S. government is in group 3 (caveat that there are lots of competing voices here and I have a very hard time modelling Trump). I have no idea where China or other world powers are.
The groups are more of a spectrum than binary. Someone in group 2 who thinks the status quo is very negative will be willing to spend more resources on a pause than someone who more loosely prefers a pause.
So I think the question of feasibility is how feasible is it for people in groups 1 and 2 to get the U.S. government into group 2 and strongly enough in group 2 that it is willing to expend lots of resources on a treaty (and I think since there are other players in group 3 you would need lots of resources). I don’t have a rock solid case that this is infeasible (I don’t think you ever really could), but I think it likely is at this time. EDIT: Adding in a little more here because I saw your conversations with others and think this may be our central disagreement. I think many people overrate how politically salient AI is. Anti-AI sentiment is all over the place, but I think its a mile wide and an inch deep. When you look at Gallup polling of the most important issues facing our country, AI doesn’t show up. I think any politicians that did serious damage to the U.S. economy and potentially started wars to pause AI would be electorally punished. I think if there are anti-AI legislation that is electorally valuable it will probably be surface level stuff that doesn’t move the needle on AI risk (e.g. data center water regulations). Now voters are not an immovable force, but I think it will be hard to move them quickly and may take some other shock to the system (impactful warning shot, widespread job loss, etc.)
As for why we might want to wait for a pause if AGI is further away, If we could have a strong pause now, I’d definitely agree with you. However, I think if you were to push the world into a weakly enforced pause it may be harder to get a strong pause down the line. In the status quo I hope we can slow down capabilities work, build up chip control systems and other monitoring options and as AI gets more powerful (hopefully before we all die) people will move towards a real pause. My concern is that a weak pause drives AI development underground, differentially hurts safety, and doesn’t allow people to update in the direction of a real pause. Like a think a world where AI development is nominally illegal, but the Chinese and U.S. Governments both had well funded secret programs is much worse than Evan’s proposal and likely worse than the status quo.
FWIW if I were a world dictator I would implement a global pause followed by heavily regulated and monitored technical alignment research. I also weakly hold the belief that people who think we are in an emergency situation should state that clearly and strongly, but have found some arguments for being more strategic compelling (e.g. Holden Karnofsky on the 80,000 hours podcast).