Many people take it for granted that government won’t do anything to address societal scale AI risk unless or until there is a catastrophic “warning shot,” where an AI goes rogue and causes some serious damage. Something like Chernobyl or 9/11, where a bunch of people die.
Many people have told me they hope for such a warning shot. This is grim.
Fortunately, I don’t think we need a warning shot. Why not? Well, here are a few reasons:
My personal experience over the past >15 years is that over time, more and more people become more and more concerned about the problems. This might not happen fast enough, but it’s been very fast since the start of 2026.
Job loss or other societal effects of AI could create political will to stop AI, even absent any loss-of-control type catastrophe.
It seems like the problem is not the people don’t care, it’s that they aren’t paying attention and/or don’t understand the situation. So things that draw attention to the issue, including less harmful warning shots like the Hugging Face Incident, but also deliberate efforts like the Statement on AI Risk or Pacing the Frontier can move the needle and, ultimately, be sufficient.
People regularly get all up in arms about things that don’t matter as much or aren’t even real issues.
But I’m writing now to put forth a new theory: The examples people are comparing AI risk with are different. They are using a bad “reference class”.
The main difference is: A single AI incident could kill everyone, and it is far from guaranteed that we would get such a warning shot in advance. If people — including politicians — realize this, they will want to act to prevent this.
With the other examples people point at, politicians mostly could afford to wait until there was a warning-shot disaster. Their lives weren’t clearly on the line. They may even have preferred not to act, and just had their hand forced by the public outcry.
But AI risks threaten politicians and those with power. If they realize this, we should expect them to view it like a possible war with a rival power: extremely seriously.
Perhaps the best comparison in this regard is nuclear weapons, where governments seem to have quickly realized their destructive potential, before seeing a single detonation. After WWII, the US and the USSR both agreed (at least ostensibly) that they wanted complete nuclear disarmament, they just didn’t agree on the details.
Or, there’s the example from fiction: an alien invasion.
If you think that people need to see a catastrophe to understand that AI is dangerous, you must not think much of their intelligence. It’s true that a lot of people aren’t paying much attention, or are being misled by false narratives. But those are fixable problems, and we don’t actually need a bunch of people to die to fix them.
If you think we do, imagine explaining to people: “We knew you were at risk, but we just figured you were too stupid to understand that unless we let a bunch of you die.”
Thanks for reading The Real AI! Subscribe for free to receive new posts and support my work.
Well said. The first half is a background argument for my thesis that The 2028 presidential primaries could be crucial for AI outcomes. I didn’t try to put this part of the thesis into that short post, so I’m linking to this.
Public opinion shifts, and it could well shift quickly on AI. Let us hope and help that happen fast enough.
A lot of this argument rests on the assumption that the public and elites will naturally come to believe in AI existential threats because they’ll logically realize that it’s important for the world or aligns with their interests. I personally don’t think that is true, because for most people, a lot of their beliefs are emotional in nature. We are built to change our emotional beliefs when something visceral happens, and it is hard for us to learn from history and yet harder from abstract predictions.
Here’s a recent LW post that shares some anecdotes about mainstream views of AI that I find concerning, but illustrates my point about a lot of public beliefs in this area being emotionally driven: https://www.lesswrong.com/posts/ySXuvJcqRindQwAk7/how-my-students-think-about-ai
So I largely disagree with this post (I agree with only point 3 out of the 4 reasons in the post). However, I don’t agree either that the world necessarily needs a real severe warning shot before people become serious about AI, because I think that while the warning needs to be visceral, it need not be real. I think a good solution is to create concrete hypothetical scenarios of AI alignment based on sound predictions, along the lines of AI 2027, AI 2040, and the scenario in If Anyone Builds It, Everyone Dies. (Another great example in video format is Slaughterbots.) I personally also think that besides text and video scenarios, a video game on the topic could be a good idea for the right audience.