It would have been relatively easy to tell a story ex-ante about how the Pentagon dispute wouldn’t hurt their revenue.
Doesn’t seem as easy to tell a story about how pausing all work related to model improvements would have the same effect.
I think announcing and following through with a unilateral pause is a one-time-only irreversible move with very low chance of cascading into a global pause
I agree there’s a lot of uncertainty here. But what are the alternatives? Do we really believe that Anthropic will solve alignment in time, while nobody else (or at least not whoever surpasses them) will be able to? This seems highly implausible to me. If alignment is just a matter of having a smart-enough AI working on it (which I doubt very much), then whoever will become the new leader will solve it that way (however “bad” they are, they have no interest in losing control over their AI either). If it is really hard to solve, then it doesn’t matter whether Anthropic or someone else pushes us over the cliff.
If they really believe what they say in that post, they should pause and at least try to save our future.
Accepting for the sake of argument that the only hope we have is a global pause—the question that matters is:
“Are we more or less likely to see an effective globally coordinated pause if Anthropic decides to unilaterally stop improving models tomorrow.”
This is a complicated, messy question. My impression is the answer is “less likely”.
The alternative to a pause today is to continue gaining market share, and (hopefully) leverage that in order to deepen relationships with other relevant companies and governments, continue advocating publically from a position of undeniable credibility, continue collecting the best talent in one place etc.
If Anthropic had pulled the brakes 6 months ago, I don’t think we’d feel any safer today
*P.S. I don’t have any insider knowledge at all I’m merely speculating at other’s intentions based on publically available info
It seems to me that a unilateral pause would have a higher chance of working later, when more researchers at more labs are concerned about alignment, rather than now.
It would have been relatively easy to tell a story ex-ante about how the Pentagon dispute wouldn’t hurt their revenue.
Doesn’t seem as easy to tell a story about how pausing all work related to model improvements would have the same effect.
I think announcing and following through with a unilateral pause is a one-time-only irreversible move with very low chance of cascading into a global pause
I agree there’s a lot of uncertainty here. But what are the alternatives? Do we really believe that Anthropic will solve alignment in time, while nobody else (or at least not whoever surpasses them) will be able to? This seems highly implausible to me. If alignment is just a matter of having a smart-enough AI working on it (which I doubt very much), then whoever will become the new leader will solve it that way (however “bad” they are, they have no interest in losing control over their AI either). If it is really hard to solve, then it doesn’t matter whether Anthropic or someone else pushes us over the cliff.
If they really believe what they say in that post, they should pause and at least try to save our future.
Accepting for the sake of argument that the only hope we have is a global pause—the question that matters is:
“Are we more or less likely to see an effective globally coordinated pause if Anthropic decides to unilaterally stop improving models tomorrow.”
This is a complicated, messy question. My impression is the answer is “less likely”.
The alternative to a pause today is to continue gaining market share, and (hopefully) leverage that in order to deepen relationships with other relevant companies and governments, continue advocating publically from a position of undeniable credibility, continue collecting the best talent in one place etc.
If Anthropic had pulled the brakes 6 months ago, I don’t think we’d feel any safer today
*P.S. I don’t have any insider knowledge at all I’m merely speculating at other’s intentions based on publically available info
It seems to me that a unilateral pause would have a higher chance of working later, when more researchers at more labs are concerned about alignment, rather than now.