Developing options for coordination around a possible ban/pause in the future is not actual endorsement of a pause. People doing useful work on coordination could disapprove of support for actual pauses at this point, in which case it’s mistaken and impolite to call them pause supporters.
the way you do coordination is by saying you would like X if only everyone else also wanted X. you ease yourself into it, making tiny steps and ceding imperceptibly small pieces of ground until coordination has already been accomplished
Coordination is distinct from options for coordination. Options can be useful even if you believe that the best outcome involves never exercising them. Submarines with nuclear warhead missiles hopefully aren’t there to ease the world into a nuclear armageddon. People who expect good outcomes might say that an AI ban/pause is by default a catastrophe in the sense of opportunity cost, but there could be future circumstances that make it necessary, and so it’s prudent to develop the option.
Why would someone work (/ pay for working) on options for possible future coordination around a pause if they disapprove of the pause? What would they expect to gain?
Also, even developing options without endorsement (/ with dis-endorsement) indicates that one considers pause potentially viable in the future, which is a massive change from the situation, e.g., one year ago, when Anthropic and OpenAI clearly considered it not worth their thought.
From the position of the present state of knowledge, some people don’t want a ban/pause button to be pressed now, but they want the button to be there in case they want to press it in the future. I don’t think saying someone “doesn’t endorse an AI ban/pause” should by default mean they won’t change their mind when the fabric of reality is on fire, or can’t at all anticipate the possibility.
The disagreement is rather that they assign a low probability to extinction/disempowerment on the current trajectory, expect concrete and actionable signs of things going very wrong, and think that the button will still work if it’s not pressed much earlier than that.
It seems like an even more useful thing to do would be to come up with clear conditions under which they would support a pause (so not just RSI but a definition of RSI). Obviously this is really hard! But it seems like an obvious point of failure here is that the labs all agree on a conditional pause but the condition is something unverifiable, vague or very low probability (e.g. we’ll pause if there’s an AI disaster that kills thousands).
Now seems an especially low-cost time for AGI company employees to make statements in support of pauses.
Over the last week, there has been an unprecedented level of support for a pause/slowdown from AGI companies. A few examples:
The Anthropic RSI blog post
OpenAI’s “our plan” blog post
Roon (OpenAI) on Twitter
Jasmine Wang (OpenAI) on Twitter
Stephen McAleer (Anthropic) on Twitter
Developing options for coordination around a possible ban/pause in the future is not actual endorsement of a pause. People doing useful work on coordination could disapprove of support for actual pauses at this point, in which case it’s mistaken and impolite to call them pause supporters.
the way you do coordination is by saying you would like X if only everyone else also wanted X. you ease yourself into it, making tiny steps and ceding imperceptibly small pieces of ground until coordination has already been accomplished
Coordination is distinct from options for coordination. Options can be useful even if you believe that the best outcome involves never exercising them. Submarines with nuclear warhead missiles hopefully aren’t there to ease the world into a nuclear armageddon. People who expect good outcomes might say that an AI ban/pause is by default a catastrophe in the sense of opportunity cost, but there could be future circumstances that make it necessary, and so it’s prudent to develop the option.
Why would someone work (/ pay for working) on options for possible future coordination around a pause if they disapprove of the pause? What would they expect to gain?
Also, even developing options without endorsement (/ with dis-endorsement) indicates that one considers pause potentially viable in the future, which is a massive change from the situation, e.g., one year ago, when Anthropic and OpenAI clearly considered it not worth their thought.
From the position of the present state of knowledge, some people don’t want a ban/pause button to be pressed now, but they want the button to be there in case they want to press it in the future. I don’t think saying someone “doesn’t endorse an AI ban/pause” should by default mean they won’t change their mind when the fabric of reality is on fire, or can’t at all anticipate the possibility.
The disagreement is rather that they assign a low probability to extinction/disempowerment on the current trajectory, expect concrete and actionable signs of things going very wrong, and think that the button will still work if it’s not pressed much earlier than that.
obviously the glorious international pause treaty is a good idea, and i think people have always overestimated the cost of saying so publicly.
It seems like an even more useful thing to do would be to come up with clear conditions under which they would support a pause (so not just RSI but a definition of RSI). Obviously this is really hard! But it seems like an obvious point of failure here is that the labs all agree on a conditional pause but the condition is something unverifiable, vague or very low probability (e.g. we’ll pause if there’s an AI disaster that kills thousands).