AI Safety Can’t Afford a Second Cause
Imagine an astronomer who discovers an asteroid with a 50% chance of hitting Earth in 2035. She goes on TV. She testifies before Congress. She founds the Asteroid Deflection Institute and starts doing fundraising rounds. And then, in between appearances, she prolifically tweets about zoning reform.
I think that most people agree that this would be pretty weird. Her zoning takes might even be correct, but why would you do that in the first place? If she has attention left over for zoning, how real can the asteroid be? Thus, the asteroid stops being filed under “horrifying, world-uniting emergency” and starts being filed under “political cause”.
This essay is obviously about AI safety. If you believe that AI is the biggest existential risk that humanity is facing in the 21st century, then you believe that you have located the asteroid. Almost everything else you care about is downstream of the asteroid not hitting. And yet the AI safety movement is full of people who profess this belief but also produce a steady stream of public commentary on housing policy, (unrelated) geopolitics, campus culture, and whatever else drifted across the timeline that week. If you don’t believe me, just look at leading AI safety figures’ accounts on Twitter.
I think that this is a mistake, and I want to explain why.
First of all, I don’t want to argue that AI safety should stay out of politics. That would be stupid: AI safety is politics. You can’t “stay out of politics” while asking governments to regulate compute, control exports, impose liability on the richest companies on Earth, and sign treaties with geopolitical rivals. An AI safety movement that tried to stay out of politics would consist of people writing alignment papers nobody reads.
What I am arguing is that AI safety should stay out of non-load-bearing politics (from now on called “hobby politics” for simplicity). Load-bearing politics is the stuff you can’t avoid because it is literally what you came here for: compute governance, licensing, treaties, and so on. Hobby politics is everything else — your takes on the Middle East, on immigration, feminism, whether San Francisco should build more housing, and so on.
The load-bearing politics is the mission. The hobby politics is a tax on the mission, paid from the “credibility with people who don’t already agree with us” account that you share with the rest of the movement.
So here is my thesis: AI safety should do the politics it must do to achieve its central aims (like “humanity doesn’t die” and “humanity is not fully disempowered”) and not one take more. In other words: AI safety cannot afford a second cause.
But surely this is not needed? After all, AI safety can avoid polarization by framing itself around universal values. Surely, “we don’t want everyone to die” polls pretty well across most demographics these days.
This sounds compelling until you look at what happened to climate change.
Climate change had the most universalist framing imaginable. The planet is on fire, we need to save it to protect the future of our children. For God’s sake, they had polar bears. But the polar bears didn’t save them from polarization.
Why? First, because the solutions threatened specific interests: carbon taxes and drilling restrictions would hit certain industries, and so those industries spent decades and billions deliberately coding climate as a left-wing hobbyhorse. A polarized issue is a stalled issue, after all. Second, the proposed solutions happened to rhyme with the left tribe’s preexisting preferences like regulation, international coordination, constraints on industry, so the climate change movement was nicely absorbed into the greater “left vs. right” tribal fistfight.
And the movement was not nearly disciplined enough to avoid this. Instead, climate change activists weighed in on every cause du jour imaginable. So persuadable-but-skeptical Republicans got confirmation of the suspicion the fossil lobby spent decades planting: this was never about the climate; this is a package deal, and it is a package I dislike.
The lesson from climate change is not “universal framing keeps you safe, unless someone is undisciplined.” The lesson is: bipartisanship is an unstable equilibrium that hostile actors will actively try to push you out of, and maintaining that equilibrium will require deliberate, continuous effort. It will require cultivating champions across the aisle. It will also require framing that gives each tribe a native reason to care. And it will require not handing your prospective opponents ammunition for free.
We should keep in mind that the rationalist-adjacent / AI safety community is selected for people who love to argue. Debating is fun. Asking a rationalist to see a wrong opinion on the internet and not correct it is like asking a border collie to watch sheep walk by. The discipline requested is real discipline, and the movement has, let’s say, an uneven track record with impulse control around Being Right in Public. But the virtue of silence is a real virtue.
And AI safety’s asks rhyme uncomfortably with the existing package: “regulate tech companies” sounds left, “beat China” sounds right, and both tribes have noticed. Already, AI safety is starting to be seen as kind of left-coded (if you really think that the Grey tribe is a thing in broader politics, you are severely mistaken regarding people’s ability and will to distinguish the Grey tribe from the Blue tribe). This could become a big problem given that Republicans currently hold power, and that even if they lose the next election, their support will matter a great deal going forward.
How do we avoid this kind of polarization? At the very least, we will need to stick to three simple rules:
Formalized organizational nonpartisanship. Safety orgs should adopt the standard think-tank posture: no endorsements (except for perhaps AI-safety-related ones), no institutional opinions on anything that isn’t the asteroid.
A cultural norm that the more public-facing your safety role, the more your pet causes stay in the drawer.
Active and deliberate cultivation of champions on both the left and the right, because universalist framing alone won’t do it for you.
This will not be easy, but I think it is very much worth doing to avoid AI safety going the way of climate change.
The left’s view on AI has been decided for years: don’t use AI, don’t help build it, and suppress it with regulation as much as possible. I agree with this view. On the right I don’t see anything comparable. You give the example of “beat China”, but this is used to promote the arms race more than to promote safety.
Moreover, I think the results of AI safety-minded people being allergic to the left have been bad. It allowed these people to join frontier labs as enablers. These labs ended up working with the US government and racing with each other and the world. If safety-minded people had been part of the left “tent”, they’d have been more wary of the commercial motive and would’ve worked against AI on the public side instead.
I disagree that the left’s view on AI is “don’t use AI, don’t help build it, and suppress it with regulation as much as possible”. The left’s view on AI these days is much more “AI is a scam”. I don’t think it’s true that the left uses AI because of a virtuous decision to unilaterally disarm, the left doesn’t use AI because it doesn’t believe it works.
This is also why the AI safety movement is (correctly) reluctant to throw its weight behind the left (combined actions like https://stoptherace.ai/ notwithstanding). Such a decision would align the safety movement with a camp that disagrees on the core premise—namely that AI is extremely transformative. And, quite honestly, if the main problem with AI will be that it will simply not really work and there will be a bunch of scams around it, then worrying about AI just isn’t all that important (this would be the “AI as bitcoin” world).
I also disagree on the point that left people joined the labs as “enablers” en masse. Most people working for AI labs seem to be your garden-variety mostly apolitical tech nerds that like working on cool stuff.
The left is a big tent. Here’s two writings on AI by people who are active and respected on the left: Mike Monteiro saying there will be “a beacon in space roughly where Earth used to be”, and Sam Kriss saying AI “might kill every single person on the face of the Earth”. There are plenty of people on the left with such views, and if AI safety-minded people joined, there’d be even more.
It’s true that there’s a strong focus on present harms. But that’s just a fact about the left, they are attuned to harms that are actually happening. To use that as a reason to decline alliance with the left, and instead go “bipartisan” and court the right who are all “go industry, go war” while ignoring all harms present and future, seems very wrong to me. I think it’s been one of the biggest missed chances in our community, and I’m hopeful we can still change course on this.
Regarding your last paragraph, I think maybe you misread a bit. My point was that many safety-minded people, attracted by writings on LW and other places and not having a leftish “immune system”, decided to join for-profit AI labs to work on alignment. It might seem harsh to call them “enablers” for that, but I stand by that characterization. See more on my reasoning in this comment.
I agree that the left is a big tent and there are always examples of every opinion present in every political movement, but I mostly agree with this take: https://www.transformernews.ai/p/the-left-is-missing-out-on-ai-sanders-doctorow-bender-bores
This doesn’t take away from the correct argument that there are prominent leftists who are very vocal about existential AI risk, including really prominent people like Bernie Sanders. But for the most part, the left movement is focusing (as you say yourself) on the more mundane harms. This is not necessarily bad, but it means that there is a fundamentally different focus.
As for the right, it is interesting that you (correctly) insist that the left is a “big tent” with many opinions, but then the right is a uniform bloc that wants to “go industry, go war.” There are plenty of Republican politicians seriously worried about AI. Just look at Josh Hawley, Marsha Blackburn, Marjorie Taylor Greene, and Ron DeSantis. Are we really sure that the AI safety movement has no touchpoints with people who say things like “we have no idea what AI will be capable of in the next 10 years, and giving it free rein and tying states’ hands is potentially dangerous”? Because I am not.
Even Trump himself stated in 2024 that AI is “scary” and “dangerous.” Yes, his current policy does not support this view, but I press “F” to doubt that there is absolutely no way the Republicans can be reached on this issue. I agree that it will be very hard, but very hard problems are the whole business of AI safety.
Being able to do outreach to Republicans has a very important pragmatic reason, by the way: they are the ones in power right now, and they will be in power until at least 2028 (even if the midterms reshuffle the board a bit). Not to mention that it is absolutely not a given that the Democrats will win the 2028 election. Therefore, if you think that something like https://ai-2027.com/ is even remotely reasonable, then Republican support will be extremely important. And if not, then there is a fundamental difference here between “AI is the most transformative technology ever, and it might/will likely kill everyone if left unchecked, and it might happen very soon” (AI safety movement) and “AI will produce nasty deepfakes and do various other mundane harms” (left movement). That doesn’t take away from the deepfakes! They are nasty! But there is a large difference between “might end the world” and “will do various nasty things that we will have to deal with as a society but will definitely not end the world.”
The two writers you quote are not really evidence of leftist receptivity to the idea of AI as an extinction risk. In both cases, they are focused on AI as a cause of cultural degradation, and are simply giving a rhetorical nod to a vague maximum dystopia in the midst of their rant. It’s not like they are making a list of AI threats they take seriously, and then noting, oh and by the way, the AIs might take over and wipe us out, too. Their notion of long-term risks from AI is more like dysfunctional Idiocracy or farcical techbro dictatorship.
Great first post. (Even if I don’t understand all the ‘left’ and ‘right’ stuff.)
Your argument seems compelling, but let’s test it against two activists, Greta Thunberg and Richard Stallman. [1]
On the one hand, Thunberg’s climate activism seems to have gone off the rails. In the past, Thunberg presented climate change as an existential threat (or something very close to that), and made extreme changes in her own life consistent with that. But now, Thunberg seems to have more than diluted the climate issue; she seems to have dropped it entirely.
I have no idea what happened. Did she change her mind? Did she continue to believe that climate change was an existential threat but somehow lose interest anyway? This vaguely bothered me even before reading your post.
On the other hand, it’s perfectly clear what Stallman believes in. In particular, even if Stallman’s primary focus lies elsewhere, it’s perfectly clear that Stallman also believes that climate change is an existential threat. [2] That remains clear despite the fact that Stallman also prolifically posts about all sorts of other things. [3]
When I see those other posts, I don’t think Stallman has forgotten about or devalued the climate issue, I just think he’s done as much as he can already, so he’d might as well spend his remaining time on other important issues. I also think he has a healthy balance (at least as far as these issues are concerned): if you’re going to fight for decades to save the world, you should also spend some time making sure that world is worth living in.
Having said that, I have to admit: when Stallman posts about trivial things, like his petty disputes over terminology, I do get annoyed at his apparent distraction. So maybe Stallman is somewhat like Thunberg after all.
This choice is completely subjective; they just happen to be two activists that come to my mind.
Particularly good posts include Children from all countries exposed to climate hazards, Global heating threatens national security and ‘subsidizing energy made from fossil fuels is killing civilization’, but Stallman posts something climate-related almost every day.
This includes everything from petty disputes over terminology to child development to foreign politics to Palestine (and that’s just a small selection of items from the last few days).
It’s a spectrum where some people are less annoying and some people are more annoying, but in the end every time you talk about things unrelated to your main cause(s), you pay a tax by actively pushing away people who don’t already agree with you on everything (read: most people) for no real benefit to your main cause(s). And since AI safety is already the hardest problem humanity has ever faced, I don’t get the feeling that the AI safety movement can afford that tax.
And Stallman is a great example because the man had a lot of excellent ideas, but he was such a poor communicator that he really shot himself in the foot again and again for (I repeat) no benefit. Just look at why he had to resign as president of the FSF — you cannot look at that and reasonably say that this was good decision-making that furthered his cause(s).
Stallman is also a great example because he is a much more typical failure mode to be aware of for the average person in the AI safety movement than someone like Thunberg. That failure mode looks like this:
a) I am smarter than everyone else on technical questions
b) Since I am smarter than everyone else on technical questions, I am smarter than everyone else in general
c) Since I am smarter than everyone else in general, I can get away with things a normal human would never get away with: being very annoying, very hurtful, or focusing on ten problems at the same time
I have met a number of people in the AI safety movement who resemble this pattern, and we need to be very careful not to fall into it, but instead to think more along the lines of:
a) I am smarter than normal people on technical questions
b) It is very unclear how much that generalizes — for example, I might be terrible at policymaking questions or business questions (or whatever else)
c) Since AI safety is an incredibly hard problem whose solution has to come together from multiple vastly different domains, I will stay very focused on the problem at hand, I will remain humble and vigilant about my own opinions, and I will especially be nice, because being mean brings my cause absolutely nothing
(which, to be fair, immediately reminds you of the 12 virtues of rationality)
Many people in the AI safety movement see these causes as the same thing, or the “root cause” behind all these problems. See all the talk about coordination and also Richard Ngo’s sociopolitical thinking.
This is too oversimplified to be useful, really. There are many right-wing movements that have some environmentalist aims, and “regulation is a left wing idea” is so incredibly broad as to be basically useless. Take Reform UK, a right-wing party, having almost half its prospective voters being pro-climate-action, and half its prospective voters being against it (CTRL + F for “climate” in the data table). Intuitions in this space are too over-indexed on surface-level news in the last 10 years and it’s leading them astray IMO.
If you wanted to take a lesson away from environmentalism, perhaps it’s that AI safety will split along nationalist and internationalist lines. Which I suppose we’ve seen some examples of already.