It seems important that they paused the Mythos release based on specific and easily understood concerns, and it seems to be holding up that they were right. I’m not sure a pause based on vague concerns followed by restarting after not making much progress would send the same signal.
I agree that Mythos is a much clearer case, but the stakes were much lower. I mentioned it to show that AI labs can send strong signals and change a lot in politics, so “we can’t do anything, because if we don’t do it, someone else will, we won’t be at the forefront any longer and nothing else changes” is clearly wrong.
To me, there’s nothing “vague” about losing control of an advanced AI. We don’t know what the specifics are, how and when this is going to happen. But if we knew, it would be too late. We do know that as long as we haven’t solved alignment, it is suicidal to build an uncontrollable AI, and this is where Anthropic is heading.
Yes, it isn’t clear at all that such a bold move as a unilateral pause would achieve anything. But that is no excuse not to try it if we don’t have any better idea how to prevent a catastrophe and all of humanity is at stake.
As a father of three, I find it really hard to prevent myself from getting much more emotional in my writing.
My disagreement here isn’t about whether AI labs should stop developing AGI. I just don’t think pausing would send the same signal to outsiders without the clear (partially solveable) situation they had with Mythos.
It seems important that they paused the Mythos release based on specific and easily understood concerns, and it seems to be holding up that they were right. I’m not sure a pause based on vague concerns followed by restarting after not making much progress would send the same signal.
Pausing a release is much less important than pausing development.
I agree that Mythos is a much clearer case, but the stakes were much lower. I mentioned it to show that AI labs can send strong signals and change a lot in politics, so “we can’t do anything, because if we don’t do it, someone else will, we won’t be at the forefront any longer and nothing else changes” is clearly wrong.
To me, there’s nothing “vague” about losing control of an advanced AI. We don’t know what the specifics are, how and when this is going to happen. But if we knew, it would be too late. We do know that as long as we haven’t solved alignment, it is suicidal to build an uncontrollable AI, and this is where Anthropic is heading.
Yes, it isn’t clear at all that such a bold move as a unilateral pause would achieve anything. But that is no excuse not to try it if we don’t have any better idea how to prevent a catastrophe and all of humanity is at stake.
As a father of three, I find it really hard to prevent myself from getting much more emotional in my writing.
My disagreement here isn’t about whether AI labs should stop developing AGI. I just don’t think pausing would send the same signal to outsiders without the clear (partially solveable) situation they had with Mythos.