AI 2027 was a prediction, not a recommendation. We correctly predicted that the US government and leading companies would want to Beat China with AI and so forth. We could have inserted some wishful thinking into our scenario, but that would have been dishonest and inaccurate.
Edit to add some clarifying info: --I don’t remember ‘leaning in’ to the race framing any more than I had to to be accurate. In fact I was leaning away from it when I could, because I thought the whole time that it was going to lead the US down a dark path (i.e. the path described in AI 2027.) In some ways AI 2027 was a reaction to Situational Awareness, which used the race framing in a more gung-ho, approving way. --I do agree that AI governance is a very fraught field and a lot of it carries great uncertainty about whether helping or harming, and I agree this applies to AI 2027 and everything I have done basically. I do worry about this a fair amount. --I think AI 2040 has the same framing as AI 2027 actually, and if it seems different to you maybe that’s because you were implicitly thinking of AI 2027 as more of a recommendation rather than a prediction? Or as a recommendation rather than a warning? --Also, what do you mean by framing in this context? And what do you mean by national security? Like, do you disagree that the companies are on a path to building ASI and that if they do it’ll be the biggest thing that ever happened to the national security world / to the security of the US ever? Do you disagree that whoever controls ASI will be in a position of immense power over whoever doesn’t control ASI? Etc.
Hi Daniel, thank you for sharing. I’m glad that you didn’t deliberately lean into the race framing and I apologize for having misinterpreted your intentions.
In that case, I think my disagreement is largely epistemic; I agree with Matrice’s comment that China has not yet “woken up” to AGI, and that the race seems largely one-sided so far. The slowdown ending that AI 2027 discusses (where the largely-aligned American AI does a value handshake with the misaligned Chinese AI) is understandable in that you think the most likely outcome is the race ending and the most likely outcome where humans remain in control involves the US taking an expensive detour into alignment work (which is justified by them being farther ahead). I personally am a bit more optimistic about China’s stances on AI safety? I’m curious how your views on this have changed since you wrote AI 2027.
No need to apologize you didn’t do anything wrong. But thanks. :)
I agree the race is one-sided so far. Judging governments by their words, China is more pro-AI-safety than America. Judging by their actions too I guess insofar as China isn’t racing, though that doesn’t mean much right now because the most likely explanation is not that they see the dangers of ASI and are unilaterally refraining, but rather just that they don’t realize ASI is on the horizon yet.
I’m not sure what we disagree on actually? Can you say more about what you think will happen?
if it seems different to you maybe that’s because you were implicitly thinking of AI 2027 as more of a recommendation rather than a prediction? Or as a recommendation rather than a warning?
I think the relevant term here is “self-fulfilling prophecy”, or perhaps “hyperstition”.
I agree that self-fulfilling prophecies are a thing but I don’t think that this whole “AI is very important for national security” meme can be blamed on us. What do you think we should have written in our scenario instead?
Didn’t you note yourself in your last update that the prediction of “China waking up to AGI” by mid-2026 in AI 2027 hasn’t been met at all? In fact, the Chinese AI space is more competitive than ever, with the assent of Xi himself making a whole speech supporting openness and global governance over securitization. The arms race narrative that is helping the industry justify not slowing down is wholly one-sided and self-propelled.
I agree that we could have predicted China somewhat better by predicting that they wouldn’t think of themselves as being in an AI race in 2026, whereas instead we thought that by this point probably they would. But we would have horribly mispredicted the US if we had said that the US government and/or companies didn’t see themselves as being in an AI race in 2026.
I agree that the arms race narrative that is helping the industry justify not slowing down is wholly one-sided and self-propelled, and if you had asked me two years ago I would have said it myself. We had this in mind while writing AI 2027, and it’s illustrated in the story, e.g. the fact that Chinese actions are a response to US actions and the fact that China tries to come to the table and negotiate and the US rebuffs them.
AI 2027 was a prediction, not a recommendation. We correctly predicted that the US government and leading companies would want to Beat China with AI and so forth. We could have inserted some wishful thinking into our scenario, but that would have been dishonest and inaccurate.
Edit to add some clarifying info:
--I don’t remember ‘leaning in’ to the race framing any more than I had to to be accurate. In fact I was leaning away from it when I could, because I thought the whole time that it was going to lead the US down a dark path (i.e. the path described in AI 2027.) In some ways AI 2027 was a reaction to Situational Awareness, which used the race framing in a more gung-ho, approving way.
--I do agree that AI governance is a very fraught field and a lot of it carries great uncertainty about whether helping or harming, and I agree this applies to AI 2027 and everything I have done basically. I do worry about this a fair amount.
--I think AI 2040 has the same framing as AI 2027 actually, and if it seems different to you maybe that’s because you were implicitly thinking of AI 2027 as more of a recommendation rather than a prediction? Or as a recommendation rather than a warning?
--Also, what do you mean by framing in this context? And what do you mean by national security? Like, do you disagree that the companies are on a path to building ASI and that if they do it’ll be the biggest thing that ever happened to the national security world / to the security of the US ever? Do you disagree that whoever controls ASI will be in a position of immense power over whoever doesn’t control ASI? Etc.
Hi Daniel, thank you for sharing. I’m glad that you didn’t deliberately lean into the race framing and I apologize for having misinterpreted your intentions. In that case, I think my disagreement is largely epistemic; I agree with Matrice’s comment that China has not yet “woken up” to AGI, and that the race seems largely one-sided so far. The slowdown ending that AI 2027 discusses (where the largely-aligned American AI does a value handshake with the misaligned Chinese AI) is understandable in that you think the most likely outcome is the race ending and the most likely outcome where humans remain in control involves the US taking an expensive detour into alignment work (which is justified by them being farther ahead). I personally am a bit more optimistic about China’s stances on AI safety? I’m curious how your views on this have changed since you wrote AI 2027.
No need to apologize you didn’t do anything wrong. But thanks. :)
I agree the race is one-sided so far. Judging governments by their words, China is more pro-AI-safety than America. Judging by their actions too I guess insofar as China isn’t racing, though that doesn’t mean much right now because the most likely explanation is not that they see the dangers of ASI and are unilaterally refraining, but rather just that they don’t realize ASI is on the horizon yet.
I’m not sure what we disagree on actually? Can you say more about what you think will happen?
I think the relevant term here is “self-fulfilling prophecy”, or perhaps “hyperstition”.
I agree that self-fulfilling prophecies are a thing but I don’t think that this whole “AI is very important for national security” meme can be blamed on us. What do you think we should have written in our scenario instead?
Didn’t you note yourself in your last update that the prediction of “China waking up to AGI” by mid-2026 in AI 2027 hasn’t been met at all? In fact, the Chinese AI space is more competitive than ever, with the assent of Xi himself making a whole speech supporting openness and global governance over securitization. The arms race narrative that is helping the industry justify not slowing down is wholly one-sided and self-propelled.
I agree that we could have predicted China somewhat better by predicting that they wouldn’t think of themselves as being in an AI race in 2026, whereas instead we thought that by this point probably they would. But we would have horribly mispredicted the US if we had said that the US government and/or companies didn’t see themselves as being in an AI race in 2026.
I agree that the arms race narrative that is helping the industry justify not slowing down is wholly one-sided and self-propelled, and if you had asked me two years ago I would have said it myself. We had this in mind while writing AI 2027, and it’s illustrated in the story, e.g. the fact that Chinese actions are a response to US actions and the fact that China tries to come to the table and negotiate and the US rebuffs them.