I thought the post was useful for making me think “now that I’m reading this story about AI Pause, it just seems pretty implausible.” Both the idea of Anthropic unilaterally pausing, and also the confused but somehow positive reaction from governments. It makes it feel more concrete that if we want a pause, we probably need to target a more specific path than “convince the labs that safety is important, so hard that one day they just stop.”
@Thomas Kwa’s story also feels a bit surprisingly positive, but more plausible than the original post.
The idea that Anthropic learns about misalignment in other labs’ models seems especially helpful, since this gives them more confidence that other labs might be open to a pause. I wonder if we could set up a more reliable mechanism than “back channels”?
Ideally, labs would publicly and immediately disclose all worries of misalignment in their internal models. But since that probably won’t happen, maybe there should be private channels. Like, the CEOs can press a button saying “I’m not concerned enough about misalignment right now to unilaterally pause, but I’m concerned enough that I’m open to talk about a pause with the other CEOs, if they feel the same way.” And if two or more CEOs press this button, they get connected to each other. Or something smarter/more complicated than that.
It would be good for people to continue thinking this through more explicitly: what are some concrete scenarios in which METR/UKAISI/etc. contribute to a useful pause or slowdown? How likely are those scenarios?
I thought the post was useful for making me think “now that I’m reading this story about AI Pause, it just seems pretty implausible.” Both the idea of Anthropic unilaterally pausing, and also the confused but somehow positive reaction from governments. It makes it feel more concrete that if we want a pause, we probably need to target a more specific path than “convince the labs that safety is important, so hard that one day they just stop.”
@Thomas Kwa’s story also feels a bit surprisingly positive, but more plausible than the original post.
The idea that Anthropic learns about misalignment in other labs’ models seems especially helpful, since this gives them more confidence that other labs might be open to a pause. I wonder if we could set up a more reliable mechanism than “back channels”?
Ideally, labs would publicly and immediately disclose all worries of misalignment in their internal models. But since that probably won’t happen, maybe there should be private channels. Like, the CEOs can press a button saying “I’m not concerned enough about misalignment right now to unilaterally pause, but I’m concerned enough that I’m open to talk about a pause with the other CEOs, if they feel the same way.” And if two or more CEOs press this button, they get connected to each other. Or something smarter/more complicated than that.
It would be good for people to continue thinking this through more explicitly: what are some concrete scenarios in which METR/UKAISI/etc. contribute to a useful pause or slowdown? How likely are those scenarios?