(I began writing this post several weeks ago, but political events are moving much faster than I expected, so I am publishing now out of fear that otherwise the message will arrive too late to have an impact.)
I
In this post I want to explain a concept, and issue a warning based on it. But I expect the warning will be superfluous if my explanation is sufficient. If you want to convey the idea “the rattlesnake has venom in its fangs, so don’t let it bite you”, you won’t need a hard sell for the concluding advice if the listener understands the initial statement about venom.
The word for the concept I want to illustrate is partisanize, which means to align an issue with a political tribe. It is modeled on politicize, but the latter word is not useful here. It would be meaningless to say “Don’t Politicize AI Safety”: the project is intrinsically political. It involves international diplomacy, consensus-building, the willingness to sacrifice near-term economic growth for long-term human values, and a brutally difficult coordination problem. AI Safety is inescapably political, but not inevitably partisan. It’s possible that, like issues such as infrastructure or wilderness conservation, the topic will remain nonpartisan. Unfortunately, there are many issues that started out neutral and were partisanized via a process of tribal reasoning. As I explain below, I fear AI Safety is at grave risk of following the same trajectory.
I was motivated to write this essay by a recent stratospherically-upvoted LessWrong post entitled Why I Left Google Deepmind. The author, Dr Alex Turner, documents his extensive political efforts to convince Google leadership to adopt a more principled stance on issues related to AI Safety. Unfortunately, these efforts were unsuccessful, leading to Turner’s resignation from his lucrative and prestigious job at GDM. I concur with the community’s enthusiastic admiration for Turner’s willingness to make serious personal sacrifice for his beliefs. However, I also believe his post might actually harm the cause of AI Safety, by linking it to Blue tribe politics and thus partisanizing it.
A Red tribe citizen who read only a high-level summary of Turner’s story would almost certainly be strongly supportive of his sacrifice. The Red tribe is generally skeptical and hostile to Big Tech companies, which are widely perceived as extremely Woke (cynical observers claim that Big Tech adopted Wokeness as a way to distract the Left from the vast amounts of wealth inequality they created). Red intellectuals are aware of and enraged by Big Tech’s willing participation in the Woke censorship regime of the late 2010s. The American public is increasingly suspicious of AI and there is no reason to think this suspicion has a partisan skew. In general, conservatives by their very nature are resistant to change, and AI will obviously bring about enormous social upheaval. A conservative utopia has safe streets, high employment, strong familes, cohesive communities, and widespread religious participation. There is little reason to believe that AI will promote these goals, and many reasons to suspect it will be toxic to them.
Unfortunately, a conservative who read the actual post (as opposed to a summary) would almost certainly respond negatively, and this reaction would likely contaminate his opinion of AI-Safety activism in general. The reason is that Turner opens the post with a discussion of an extremely partisan topic: the killings of ICE protestors Renee Good and Alex Pretti. His initial activism was not about AI at all, but rather about cloud services: he wanted Google to refuse to work with the federal agencies who are responsible for immigration enforcement (ICE and DHS). While anti-Trump rhetoric is not the main point of his essay, in some places he uses accusatory language such as “federal agents should not be able to kill citizens in the street”—as if the agents had just walked up to two hapless bystanders and executed them in cold blood.
Most Red tribe members find this kind of emotional, logically unexamined accusations to be deeply offensive (though they of course produce similarly illogical retaliatory accusations). While I don’t want to litigate Culture War topics here, I do want to convey the Red viewpoint on this issue. Red citizens see ICE as fulfilling a democratically decided mandate to enforce immigration law — a mandate that the Biden administration, in a brazen betrayal of democratic principles, simply ignored. All types of law enforcement are intrinsically dangerous, because some people simply will not follow the law unless the threat of violence is present. Finally, conservatives champion the right of all citizens to use lethal force in self-defense. So, in the Red view of the story, the ICE officers were doing their difficult and dangerous job when they were threatened by Good and Pretti. The officers needed to make a split-second judgment call with their lives on the line; they choose to respond with lethal force. This outcome is regrettable, but it hardly justifies the kind of moralistic insults and accusations thrown around by the Left. Certainly it does not justify Turner’s original crusade to convince Google to stop providing cloud services to the federal government.
II
Noted LessWrong contributor, AI researcher, and civilizational theorist Richard Ngo recently posted on the LW shortform about a job offer that had been rescinded:
I had a long conversation with Jesse and Dan, and later a long conversation with Geoffrey. In the latter I learned that, although the decision was prompted by me sending them my curriculum, Geoffrey in particular was more concerned about my tweets. I’ve tweeted some fairly controversial things over the last few years… It seems like he’s excited about hiring academics who are new to alignment, and worried that those people might not join if they hear about such tweets… Geoffrey claimed that he would have made the same decision if I’d posted equivalent left-wing takes.
Ngo reports that one of the Resolution executives cited this tweet as being particularly objectionable. I’ll quote just a section, which occurs to me as plausibly the most inflammatory:
The tech left views (the country) as a charity. To them, wanting others not to receive what you’re been given is hypocritical. This view is also empty, because it doesn’t tell you where the windfall actually comes from—and once you start to talk about the benefits of culture, ethics, institutions, etc, it becomes clear that citizens (and their ancestors) built that windfall rather than just being given it.
For someone on the tech left, whose alleged viewpoint was being critiqued, I can see how this comment has a lot to complain about. Ngo suggests that the citizens of a prosperous country deserve their windfall of wealth, education, safety, healthcare, etc, because the conditions that produced those goods were created by themselves or their ancestors. But it’s self-evident, from a brief review of any social media app, that modern societies have millions of people who don’t contribute very much. And given the ongoing debate about the ethics of inheritance, it’s not clear why modern people should reap such enormous benefits from their ancestors’ efforts.
While it’s completely valid to argue against Ngo’s tweet, the point of mentioning it is to highlight that he cannot possibly be considered so far out-of-bounds politically as to make rescinding a job offer a reasonable response. He did not call for political violence or hurl profane insults at anyone. I see dozens or hundreds of far more offensive comments by Americans of all political persuasions during my daily surf of X.com and Facebook.
III
One of the core insights of the rationality movement is that people are far more inclined to reason based on images and associations than real logic and argumentation. Suppose a person is faced with the question of whether to support policy X. A naïvely optimistic view of human nature is to suppose that the person will find arguments and evidence that bear on the question of whether policy X is helpful, and decide based on weighing this evidence.
The vast majority of humans simply does not follow the rational approach. Instead, they look around to see what other members of their tribe are saying, and adopt that opinion as their own. This process of observation and imitation happens continually and rapidly, since new social questions emerge all the time. It is important from a social-strategy perspective to avoid making a faux pas by expressing approval of an idea the tribe has already decided to condemn, or vice versa.
Since the observation-imitation process happens so rapidly, people have learned to take shortcuts. A full assessment of another individual’s tribal loyalties is usually too time-consuming to perform regularly. Fortunately, there are a wide variety of superficial characteristics that can be used to infer this information. Hair color, sexuality, hobbies, media preferences, fitness habits, career choices, and education levels all correlate significantly with tribal affiliation. While each individual correlation might be weak, you can produce a quite powerful prediction by combining several of them together (this is the idea of the classic ML technique called boosting). This works quite well in practice: if you meet a blue-haired librarian in Berkeley, you can be quite confident that she did not vote for Trump.
So normies are constantly observing people at a superficial level, gauging their tribal affiliation, and using the information gained thereby to guide their thinking on sociopolitical issues. On some technical topics, we might trust someone who is tribally misaligned: the blue-haired librarian might trust a Texan redneck to fix her car. But for politically impactful issues, we are going to trust people who look like us and act like us.
Given this fact, suppose a normal conservative is shown the following image, and told that the people in the image support policy X. How will their opinion about policy X shift as a result of viewing the image?
At this point, I want to invite rationalist, Blue-aligned readers who are concerned with AI Safety to take a step back and view the material I’ve mentioned above through the lens of tribal reasoning. Think about how a typical conservative, who has no understanding of AI Safety or the culture that surrounds it, might respond to these messages. Here’s my take:
(TurnTrout post) : the author is deeply Woke and also concerned with AI safety. Woke people hate me, so fuck AI Safety.
(Ngo job offer rescission) : AI Safety orgs are so deeply Woke that they would take back a job offer because of an extremely unspicy conservative tweet. Since I’m much more conservative that Ngo, AI Safety people must hate me too, so fuck AI Safety.
(Aella photo) : AI Safety people are weird Berkeley sex orgy cultists. I don’t want to be around those people, so fuck AI Safety.
To be very clear, I don’t endorse this reasoning process. I respect TurnTrout’s willingness to make sacrifices for what he believes; I don’t have a problem with weird Berkeley sex practices; I’m sure Ngo will find an excellent position that makes wonderful use of his talents. I’m simply telling you how right-leaning normies will respond, because of human tribal reasoning-by-association.
IV
I’m part of the unfortunate generation whose entire adult life has been dominated by the Culture War (the Duke Lacrosse rape hoax of 2007 is a good upper bound for the starting point). The conflict has cost me several friends, and severely weakened my relationship with my family. I’ve read enough rationalist material to understand that my tribal enemies are not actually bad people as a whole, but I also know that many individuals despise me for being a straight white dude with generally libertarian beliefs. These conflicting intellectual pressures induced me to spend many years contemplating the philosophical aspects of the war. The deepest conviction produced by this long thinking is that the closest historical analogy to the American Culture War is the European Wars of Religion that laid waste to much of that continent in the 16th and early 17th centuries.
Here’s a quick overview of the similarities (see my essay the Hatred of the Believers for a more extensive discussion). Society in both eras is held together by a unifying ideology (Christianity / liberal democracy), which both sides affirm, and which is used as a justification for the conquest of foreign nations. The ideology is preached, interpreted and defended by a moralizing class of highly credentialled professionals (priests & monks / journalists & university professors). The cause of the conflict is corruption, real and perceived, of the moralizing class (simony / DEI). The attacking side propounds a new doctrine which undermines the power and prestige of the moralizing class (Sola Scriptura / Do Your Own Research). Because people believe that the ideology protects them from unspeakable horror, disagreements about precise technical components of the ideology (transubstantiation / transgender rights) cause bitter resentment and hatred, which ironically causes people to abandon the direct stated commandment of the ideology (brotherly love / civic tolerance of dissent). The conflict is accelerated by a new information technology that the establishment cannot completely control (printing press / social media). The attacking side deploys a full spectrum of criticism, ranging from highbrow intellectual argumentation to literal shitposting. The defending side attempts to sideline its critics by shutting them out of polite society (excommunication / cancellation).
If you accept that “religious conflict” is the best reference class to use for forecasting consequences of the Culture War, there are several important implications. First, there is no rational sense in which one tribe is right and the other is wrong. No serious modern intellectual would try to argue that the Catholics were right and the Protestants were wrong, or vice versa. Pundits condemn each other as traitors or heretics against the ideology, but the precise content of the ideology is exactly the topic of debate. The sale of an indulgence is either a blasphemy against God or a perfectly acceptable means for the Church to raise funds for its good work in His service. Partisans who recognize this philosophical difficulty also advance arguments grounded in objective (non-ideological) facts. For example, a medieval Protestant might claim that his creed is superior, because Protestants had a far higher literacy rate. Similarly, modern Blue often love to observe that their side is better-educated and wealthier, and thus Blue policies lead to a healthier society. But it is simply a category error to draw far-reaching ethical conclusions on the basis of these modest correlations.
The second implication of the historical analogy is that the Culture War is going to continue for many years. The European Wars of Religion lasted for roughly 120 years (the 95 theses appeared in 1517 and the Peace of Westphalia was signed in 1648). In the wake of the 2024 election, many observers argued that Wokeness had been defeated. This is a dangerous illusion. While it is true that Trump’s return was a setback for the Woke Left, it was simply a single battle in a multi-decade struggle. The core questions of the conflict are not close to being settled. The recent success of so-called democratic socialists such as Zohran Mamdani and Darializa Avila Chevalier and the fact that (as of August 2026) Alexandria Ocasio-Cortez is the prediction market favorite for the Democratic presidential nominee in 2028, are evidence that the Left has no plans to retreat from its more extreme positions in the face of Republican electoral wins (of course the Maga faction is similarly uninterested in ceding ground). Neither side has any interest in diplomacy; both sides view the only acceptable outcome as the unconditional surrender of their enemies.
The third implication is that, while the Culture War does raise serious ethical concerns, these are utterly overshadowed by petty religious hatred. History is replete with examples of religious strife: Protestants vs Catholics in Europe, Sunni vs Shia in Islam, Hindu vs Muslim in India. From an outside view, it is clear that none of these were grand ethical struggles of good against evil. However, it is extremely difficult for someone involved in such a conflict to realize this fact. In the war of Protestants against Catholics, one of the bitterest points of contention was the principle of Sola Scriptura. Very few people today even know what this principle means, and an even smaller minority would be willing to die for it. If it were not for the particular historical accident of AGI, historians of the future would likely view the Culture War with little interest. From their future perspective, it would be just another religious and tribal conflict, where ambitious and greedy leaders manipulated the masses into hating each other for their personal political gain. Robin Hanson correctly argues that, because of cultural drift, future people will have little interest in our modern debates about transgender rights or the Second Amendment: they’ll be too busy debating their own futuristic moral conundrums.
Unfortunately, the Culture War will, in fact, have vast historical impact, not due to its innate intellectual significance, but because almost all the work of AGI development will be done within its spatiotemporal extent. This purely random historical contingency poses a dire threat to the future of humanity.
To see the danger of this accident, suppose we lived in a alternate timeline where science had made faster progress earlier in European history, so that AGI development overlapped with the Wars of Religion. Furthermore, suppose that the geographical epicenter of AGI research occurred in an fervently Catholic area, such as Rome or Paris. In this timeline, it is inevitable that the Catholic church would place enormous pressure on researchers to ensure that AGI promoted good Catholic teachings. For example, the Church might require that AGI tools must refuse to answer questions about the Bible for fear that Protestants could use the technology to learn Christian truth without depending on the priesthood. More concretely, the Wars of Religion would likely derail any attempt at political agreement to ensure that AGI development proceeds at a cautious pace. It is hard to coordinate with your tribal enemies after they’ve starved your cities to death or viciously insulted your spiritual leader, even if you happen to agree with them on some technical topics. The conflict might also precipitate an AI arms race, with the Catholics insisting on rapid development of the technology to ensure that it did not fall into the hands of the foul Protestants.
V
I want to persuade you of the following advice:
If you care deeply about a issue, don’t let it become partisanized
An issue has become partisanized when it begins to be used by one tribe to attack the other, especially if those attacks are mindless or shallow. Some examples of partisan wedges that have arisen in the last couple of decades are climate change, #MeToo, Black Lives Matter, illegal immigration, transgender rights, and vaccine mandates. Naïve advocates might imagine that partisanization is good for their cause, because it brings the topic into public consciousness. But I suggest the opposite is true: the net effect of partisanization is negative even from the perspective of people who care about the issue. Climate change is a good example of this. This issue always had a partisan skew, due to the Red tribe’s enthusiasm for masculine industrial projects like resource extraction. But this skew was not strong enough to prevent Republican leaders such as John McCain, Lindsey Graham and Olympia Snowe from supporting legislation to reduce greenhouse gas emissions through a “cap-and-trade” approach in the 2000s. Unfortunately, as the Culture War picked up steam in the 2010s, and climate issues became overwhelmingly Left-coded, hope for a bipartisan action has vanished.
A crucial fact about the wedge issues mentioned above is that most of them have little intrinsic partisan valence. Consider vaccine mandates. Trump rightly boasted about the development of the COVID-19 vaccine as one of the great accomplishments of his administration. Knowledgeable commentators generally agree that Operation Warp Speed was successful in accelerating the development of the vaccine, likely saving hundreds of thousands of lives. Furthermore, Big Pharma is a perfect villain for the Left-wing model of the world in which corporate plutocrats enrich themselves by bribing government officials to help them loot the public purse. So there is no reason a priori to suspect that vaccine mandates would ultimately become a Left-coded issue. The partisanization of this issue cost many lives, by pushing the Red tribe towards vaccine skepticism. In the eyes of the Blue tribe, these deaths were widely seen as poetic justice, but those few who care more about public health than tribal politics might view the outcome as a tragedy.
Immigration is another issue that has no innate partisan valence, and only recently became a wedge. In 1996, the official platform of the Democratic Party stated explicitly that “We cannot tolerate illegal immigration and we must stop it”. As recently as 2015, Bernie Sanders mocked open borders as a “Koch Brother proposal” in a Vox interview with Ezra Klein. In the traditional Democratic outlook, immigration is undesirable because it undermines the bargaining power of labor relative to capital. Cultural and ethnic affinity creates social cohesion, which increases support for progressive social policies such as universal health care. Immigrants from traditional societies are less likely to support progressive thinking on homosexuality and gender equality. So the fact that immigration became a partisan issue was not at all inevitable. But now that it has become partisanized, we have the worst immigration policy imaginable: ping-ponging from a complete lack of enforcement under Biden to harsh crackdowns under Trump.
How can sincere advocates avoid allowing their issues to become partisanized? I believe the secret is to practice partisan hygiene, which has two aspects. First, more obviously, advocates should ensure that their public image contains no overt expressions of tribal affiliation. As an example, Second Amendment defenders should avoid confirming the stereotype of gun owners as rural white Christians who listen to country music and talk radio, even if that stereotype actually describes them. Second, advocates should be disciplined about refusing to acknowledge the partisan skew of an issue even when it is present. It may be that many supporters of policy X also agree with policy Y, but advocates for X should ignore that correlation — even if they also support Y! For an example of how to fail at partisan hygiene, consider the following statement from Al Gore’s web page:
Entrenched, systemic racism in our country has led to disproportionate impacts of pollution on communities of color… The need for climate action is bound together with the struggle for racial equality and liberation.
This statement lends support to the Red suspicion that climate change activism is really a Trojan horse for far more wide-ranging progressive and socialist reforms.
One very good rationalist reason for refusing to see partisan valence of an issue you care about is that your own tribal loyalties may cause to you overestimate it. If you believe strongly that Policy X is good and true, and are a loyal member of Tribe T, you are likely to seriously overestimate the correlation between support for X and membership in T. Also, your lack of understanding of Tribe S may cause you to overlook significant enthusiasm for X in that population, or fail to realize that they have little incentive to support Policy Y. I believe the Blue tribe fell victim to this illusion during the intense debates about climate change in the 2010s. Blue saw Red as being (recklessly) pro-industry, and thus opposed to any policy that might hurt the energy companies, especially the large and profitable Big Oil sector. But, in fact, the most direct effect of stricter carbon policy would simply be to reduce the use of coal, because coal burning has the worst ratio of energy output to carbon emission. This reduction might actually help the oil companies, which are really oil-and-gas companies, since those two products are cleaner fuels than coal.
As I mentioned above, the AI safety issue is like immigration and vaccine mandates: it has no innate partisan valence. Conservatives, by default, will be very happy to support policies to slow or pause AI development… unless those policies become Left-coded. People who are passionate about AI safety should obsess over partisan hygiene and scrupulously avoid even the appearance of tribal affiliation.
As a closing thought, it’s already a partisan-hygiene issue that Berkeley is the epicenter of AI Safety work. I would strongly encourage new research groups to locate in more “purple” regions such as Colorado or New Hampshire. Just as it’s unwise to underestimate how simple images and associations can undermine strong intellectual arguments, don’t overlook how cheesy gimmicks can help a cause (the Trump-as-a-McDonalds-worker photo is a great example of this). I suggest someone buys Eliezer a Ford F-150 and a cowboy hat, and organizes a road trip where he drives around to remote rural towns to spread the word of AI safety. This kind of stunt could become an iconic moment in American cultural history, comparable to Ken Kesey’s cross-country bus trip with the Merry Pranksters.
While I agree that AI safety should ideally remain a bipartisan issue, a number of small details of the narratives presented in this post give me the sense that this post will have a net effect of slightly Red-partisanizing AI safety, and so somewhat fails on its own merits.
I also don’t think trying to completely suppress partisan communication will be possible as the set of people concerned about AI safety expands. Perhaps a more sustainable solution would be to “bipartisanize” AI safety (ie have multiple political figures throughout the spectrum express AI safety concerns in the language of their own tribes).
I view myself as a Red tribe partisan and diplomat: I am a member of Red, but hope to conduct diplomacy with Blue on issues like AI Safety. I did take pains to be diplomatic in this essay, but it would be counterproductive for a diplomat to conceal his own loyalties. If I concede too many points to Blue, my own tribe will reject any compromise I manage to negotiate.
I think the discussion of the value of nonpartisan discourse is worth having, and I appreciate the effort that clearly went into this post. Your reply here does seem to be in tension with this point you made in the essay:
To what extent do you think tribal signaling is productive/counterproductive in public statements and/or image? I could see a potential difference between the diplomat and advocate roles or a gradient of disclosure, but the line feels a bit fuzzy and I’m curious about your take on it.
I support AI Safety initiatives and advocacy, but I myself am not a public face of the movement.
I’m Blue or maybe ex-Blue and I found the explanations of how certain ideas appear to the Red tribe to be interesting and informative. Also the recent historical context. I had no idea, for example, that anti-immigration used to be a Blue tenet (I had only barely began paying attention to politics around 2015ish).
For what it’s worth, I found the Red perspective taken by the essay to have some value in and of itself, and to not be too all-encompassing nor combative.
Agreed; the post’s thesis is right, but the author was the wrong person to write it. It’s pretty clear that they have substantive sympathies for the conservative side of the culture war, and that those sympathies play a significant role in coloring their own reaction to the whole situation. For precisely the reasons the post explains, this will lead people who don’t share those sympathies to distrust it—and those are precisely the people who need to be persuaded.
This post actually reads as fairly neutral to me. It’s careful to use historical illustrations instead of contemporary ones and I don’t think anything screams overly biased towards one side.
But I do weakly think the recommendation at the end could be bad and lead to more polarization. (Copying Trump’s communication style is probably not what one should do if they wish to remain non-partisan...)
Most of the post discusses current political issues, with only one section being about a historical analogy. Also, the author calls themself “a Red tribe partisan and diplomat” in another comment.
As a blue team sympathizer I struggled in a few places not to get distracted by particular details which I felt a strong visceral disagreement with, but I don’t know if someone more red-sympathetic would have had similar responses to other things. This does illustrate DanB’s main point, though; that partizanship is undesirable in AI Safety discourse because people are more likely to dismiss the important points if they feel like the issue is coded for the other team.
I agree and was pretty concerned to see this post so heavily featured. I think the post itself would have indeed been better framed as exactly your second point.
And further, I think this fundamentally misunderstands that AI safety is inextricable from some level of left-coded value if those left-coded values include things like pursuing just treatment of all human beings at the systemic level. That is, Turner leaving DeepMind cannot be separated from the issue of ICE immigration agents killing Good and Pretti; it was the honest catalyst for exposing the structural issues that eventually lead to his departure. Leaving out a true and causal part of the story in order to not offend does not seem very rational to me.
I think one could argue that if an issue has a particular partisan lean, it’s beneficial to try to push it towards the opposite lean to cancel things out, if you want it to stay bipartisan. Like right now it could be valuable to create a “Conservatives Against AI Extinction” or “Conservatives Against Misaligned Superintelligence” group.
I doubt that this post will do anything to Red-partisanize AI safety, but it might Red-partisanize the idea that AI safety should remain non-partisanized. Which, to the extent that the AI safety movement is disproportionately composed of Blue tribe members, might cause it to backfire.
Yes! And I think it’s encouraging that some are trying.
It’s been interesting to see slowdown advocates worried about partisanization, when that is also a big worry of mine as an accelerationist. One of us must be wrong if we both think it hurts our chances.
It seemed clear to me that the Covid countermeasures gained significant teeth due to the (often subconscious) association between vaccine hesitancy and disliking black people. I doubt they could have been deployed with the same scope and intensity if they weren’t backed by a sense of righteous anger.
More generally, acceleration wins by default if everyone follows their local financial incentives, and the only force I’ve seen overpower capital at scale is the will of the Democratic Party of the United States.
I guess you are saying that universal support for your agenda is preferable to left-wing support only, and I can’t argue with you there. But if we assume you are going to get 50% of the population on board, then I claim you would be better off having support correlate with left-wing views—even controlling for factors like wealth and education.
I think the simple explanation is that people who think that they’re right usually think that other people will come to agree with them over time. It’s usually rare that people believe that they’re on the Right Side of Truth and Goodness but not of History.
My model is that the government loves having regulatory power over industries, so AI will become more regulated over time. Both for government-culture reasons, and structural reasons (regulation is a ratchet, the pro-regulation side only needs to win once while the opposition needs to stop them every time), anti-regulation forces lose by default. If politicians have a 50⁄50 bipartisan split on the question of whether they should regulate, the outcome will be that the pro-regulation side gets 50% of what they wanted and the anti-regulation side slowly loses ground.
I expect that the only way to avoid AI regulation is for there to be an anti-regulation party in power that treats AI regulation as a partisan threat rather than a boring procedural matter to make 50⁄50 compromises on. Once regulation is in place, local financial incentives change and acceleration may not win by default. Of course it isn’t clear how much regulation is required to change the default, and how fast it will arrive, but it’s a bold assumption to speak as if the current status quo will persist: no one at Uber expected their early unregulated status to last forever, and the government cares about AI more than Uber.
There are many examples in history, including a good number in very recent history, in which regulations on industry have been rolled back or downgraded. Or do you just mean that starting from zero regulation there is only one direction in which change can happen?
More the latter (yes I know about preemption, there isn’t literally zero anti-regulation change that can happen, but there’s very little), but also “regulation is sometimes repealed” does not contradict the notion that it is sticky and grows by default.
I think the reason ‘slowdown advocates’ are worried about partisanization is because most of those advocates recognize that in so far as the goal is to minimize the dangers posed by AI, US policy needs to thread a very fine needle.
If AI development moves forward completely unimpeded, that is a very dangerous environment for ASI to be developed in; however if government policy were to be implemented in a blindly “directional” fashion, like simply severely slowing/stopping US development without meaningully affecting international development of AI, then development would likely just continue elsewhere (such as China) in an environment that is likely just as dangerous, if not more so, for ASI development.
In other words, though the odds are slim, those concerned with AI safety are shooting for the type of widespread governmental cooperation, both nationally and internationally, that is better served by bipartisan support. For examples of the types of suggested policy to maximize those “good” outcomes, AI-2040 is a good source if you aren’t already familiar.
Why are you an accelerationist? Are you not concerned about everyone on earth being killed?
If you believe there’s low odds for doom, it’s a rational position.
(Didn’t downvote.) Honestly there are a lot of reasons, both why I support it and why the LessWrong-style counterarguments don’t land for me. It’s a big enough question that I won’t hijack this post’s comment section to lay out the case, but I might try to get a top-level post together in the next couple of months.
I would be interested to read that post. I’m very much in the opposite camp but I would desperately love some reasons to think that everything might actually be okay. :-)
If you wish to avoid partisanizing AI safety, you would do well to avoid implying that “serious modern intellectuals” excludes all Catholics and Protestants, who feel strongly that they are right and the other guys are wrong about the most important issue in the world.
(I don’t think that line does?)
I think that if you asked modern religious intellectuals whether there was any rational sense in which the Catholics were right and the Protestants were wrong, or vice versa, many would say yes, and would be offended by the premise that this position marks them as unserious.
Oh I see it now.
I’m aligned with you on the goal and the problem at hand, and it’s definitely a valuable post. I’ve written my own take on it here.
I do have one point of practical disagreement:
This contains an incorrect implication that I see a lot among non-Red Tribe people (and the occasional red or grey tribe person WRT blue tribe). The assumption is that the interests of the outgroup are fundamentally shallow or aesthetic, and that there is a combination of words that can be uttered that will cause them to abandon those interests and adopt new ones. I’d add that this is the meat behind the Trump phenomenon—Mitt Romney in jeans was condescending and repulsive; a shameless NYC billionaire speaking frankly about the costs of immigration was much more relatable[1].
To my understanding, Red Tribe’s objection to the current AI Safety lobby is that, from their view, the X-risk narrative is a motte, while the real goal amounts to preventing AI from filling the same function that the internet did from the 1990′s up to around 2018 - allowing a scattered but high-agency and high-competence group of people to upset the existing power structure by using it as a force multiplier against established institutions. Thus, while fliers and PR statements testify about X-risk, OpenAI publishes papers on how to prevent their LLM from siding with the wrong half of the public on a contentious 50⁄50 issue that both sides saw as a coup against the democratic process in opposing directions, and Anthropic’s leadership talks about how a baked-in racial bias against Whites could “address historical inequalities”. To that ends, your Catholic-Protestant analogy is very sound. The e/acc faction of the Right sees AI as a new printing press, which mitigates the effectiveness of top-down censorship and the painfulness of measures that inhibit large-scale organization, provided that efforts to turn it into a two-tiered ecosystem in which the best capabilities are reserved for surveillance and censorship are properly mitigated.
To that ends, I think an explicitly pluralist approach to the future of AI is the way to get sympathy from the Right and Center. Something that seeks out groups that might be inclined to oppose the movement and, without any catches, gives them mechanisms by which they can probe frontier models for alignment concerns at a level that isn’t currently possible through consumer APIs. There is lots of low-hanging fruit here. Most notably, the Lives Tradeoff series is a major problem that I haven’t seen any blue-ish institutions address as one. Giving people concerned about it a direct line to OpenAI and Anthropic, and the ability to run jobs vivisecting models to determine why this is occurring, seems valuable. I could see something like the following:
An AI company or think tank offers free classes on mechanistic interpretability, targeted explicitly at both parties and a broad swath of demographics from a politically neutral standpoint. The aim is to provide a baseline level of education as to how the models work, both lifting the sanity waterline of any political discussions on AI and facilitating the following steps:
A concerned interest group, perhaps anonymously, files an application or files a small fee for the right to submit weight analysis / dataset inspection / evaluation jobs on frontier models. This can be anything from novel mechanistic interpretability research they want to try on frontier models after testing it on Qwen to probing for signs of political bias in the training data, including inspections of the outputs of the base model and the model at various stages of training. An API with which these jobs can interface with the weights, internal states, and datasets of models with proprietary architectures and training procedures is provided.
These jobs are inspected to validate that they are not distillation attacks or attempts at corporate espionage, but explicitly not filtered for perceived legitimacy. They are then run, and their results are checked for anything proprietary or privacy-compromising that would require redaction before being sent back to the researchers.
In essence, this process serves as something like a FOIA act for AI. The results of all conducted research are public domain, and politicians can request that they be explained. If the staffer of a House representative from North Dakota sees that GPT-7 views North Dakotan lives as less valuable than Vermonter lives, he gets a direct line to OpenAI to solicit an explanation of how that will be addressed in the next version.
This gives interest groups not presently represented a stake in AI safety—a major right-leaning complaint I see about METR is that its employees can’t be trusted to represent Red Tribe Americans[2], and spreading out the power and responsibility of this kind of research—and building mechanisms by which talented members of every interest group can become influential by generating interesting, useful results without fear of being gatekept—would do a lot to allay that. Many of the jobs being run would be provocative or controversial in their hypothesis, but this would be the point—demonstrating that companies or AI safety NGOs are willing to sacrifice some cachet with Blue Tribe in order to prevent bad outcomes for any major subset of humanity.
As an added benefit, this can extend across national lines. China might be much less inclined to race the U.S. if they had guarantees that they could prevent the development of systems explicitly built to work against their best interests. Much of American discourse on AI—including some press releases by Anthropic itself[3] - frames it explicitly as a weapon that will be used to advance the interests of the U.S. security state, potentially at the rest of the world’s expense, which naturally precludes much of the good faith collaboration on AI regulation that would otherwise be possible. We’ve already seen many nuclear treaties with Russia disintegrate because of perceived bad faith, and taking pains to prevent the same from happening on a new front seems worthwhile.
I’ll add that I think you missed the intent and appeal of McDonalds Trump. Kamala Harris made a number of claims about her early life that were widely seen as questionable, including a story about working at McDonalds. Trump’s shift at McD’s was a jab at that—“The other candidate pretends to have worked there to seem relatable, I’ll really do it, and in my nicest suit!”. He wasn’t trying to seem relatable, he was being open about the fact that he wasn’t.
To say nothing of the rest of the world. Will a politically homogeneous group of Caucasian Oxford students ensure a good outcome for Iraqis? Chinese? Shintoists? Even if they wanted to, would they know what these groups are concerned about?
Amodei was dishonestly cast as a peacenik by both sides of this controversy. One to deride him, and the other to flatter him. The actual content of his statements seems well in line with the neoconservative position, which should be concerning both to the Americans who disagree with it and the world writ large.
When I heard Trump start to talk like AI Safety is a leftist thing ~5 days ago, I organized “Christians for AI Safety” and am also finding people for conservatives/Republicans for AI Safety groups.
At STAIR (Stop the AI Race), we had the biggest and longest AI Safety protests in history. I also helped edit/give feedback on the early IABIED draft. When all 4 labs + Bernie started calling for a slowdown, we stopped occupying OpenAI.
My parents were Republican politicians, and I sang America, the Beautiful and Proud to Be an American at Occupy OpenAI and gave a speech about keeping AI Safety bipartisan
We tried to involve Jonathan Pageau/Jordan Peterson’s “ARC” network, as they’ve been great on AI Safety for 3+ years, but it’s not clear they’ll do anything by Sep 24 when Trump and Xi talk AI
I also have the same birthday as Xi and taught english to Chinese students for years (including in Suzhou, China) and speak a little Mandarin.
My philosophy is to bridge between different groups/philosophies with love
This seems factually incorrect to me. E.g. see https://www.migrationpolicy.org/journal/policy-beat/comparing-biden-and-trump-deportation-records
The source you cite is far from nonpartisan, and is making very questionable claims. In general, the counterargument to the narrative they’re trying to set is that, when the border is open, occasional turnbacks counted as deportations overinflate enforcement activity (and, conversely, when it is closed, the lack thereof deflates enforcement activity). The same debate was had during the Obama admin. In general, the good faith consensus is that those numbers are not representative of actual enforcement, and get abused by both Democrats (claiming to be more moderate than they are) and the right flank of Republicans (claiming the current administration is softer on illegal immigration than it is) to misrepresent the actual policies being implemented.
A much more useful figure is border crossings, which track enforcement in a harder-to-game and vastly more intuitive way. Here’s a graph, courtesy of the LA Times.
There isn’t one figure you should be looking at. The much higher encounters under Biden were largely because of increases in the number of attempted crossings, more so than US policy changes. And the dropoff at the end of Biden’s term can be attributed to deals Biden made with Mexico which led to them increasing enforcement on their side of the border.
You want to look at the context and disposition of cases. IF ‘encounters’ spike but at the same time the number released (without being arrested or deported) correspondingly increases, that is indicative that many of those stops aren’t valid. Under Biden, the number of encounters went up massively and the rate of those stopped being removed stayed around the same (it increased slightly, but not categorically). That would pretty strongly indicate that Biden was at least on-par with previous border enforcement, despite having a more strained system.
Yeah the article is a lot easier for me to read when I interpret the conversations about political examples on the object level as not about truth-claims of the actual world, but of what some conservatives, including probably the author, believe.
The enemy always gets a vote.
I feel we may have an easier time convincing the Red side of AI x-risk than it might seem. Right now, it seems to me that the Blue side doesn’t really believe that AI is capable of much of anything (stochastic parrot, art theft photobasher, etc.). With Trump throwing his weight behind datacenters and advancing AI, it seems likely that Red side people will use and integrate AI more and more into their lives, and see how far along they are, and patterns indicating their lack of alignment.
A thousand little warning shots, if you will—one guy’s agent might steal his bank credentials from his password file, another lady might have her agent unintentionally post to Facebook on her behalf asking for the best muffler shops. The smallest, least harmful things, ideally, that might get people on the Red side to pause for a second.
But, otherwise, yes, 100% agreed, holy fuck do not let this issue become partisanized. You cannot go back once it is.
Another minor thing that I want to raise in this whole mess—beware the 24-hour news cycle. Visakan Veerasamy has a long-running Twitter thread documenting the various “top stories” in the news cycle, going back several years, that seemed like they dominated every corner, but then were replaced with the next one. The news cycle moves on, the stuff that’ll replace this moment of AI x-risk awareness hasn’t happened yet, but when it does, this will be yesterday’s news.
So fast motion is preferable.
(not advocating for a conspiracy of the media or anything, just that the natural flow of events and news cycles displace older topics regularly)
Thank you for writing this. I am open to, but not yet convinced by, the argument that it is best to keep AI safety nonpartisan. My main contention, though, is that even with every reasonable precaution, I am skeptical that the issue can remain bipartisan for long (I would genuinely be interested to hear if you think I am getting something wrong here). Ultimately, I see this issue becoming partisan for two reasons that I have experienced firsthand:
1. When you talk to political offices about AI Safety and you actually get them to care about the issue, they always seem to convert the issue into a political wedge to attack their political opponents. I.e. politicians are extremely liable to convert any issue from their constituents into a political issue that they can use in their political contest to get elected. For example, I recently heard that some Democratic Party leaders are already discussing the possibility of making AI safety and AI regulation a major political wedge issue in the 2028 election.
2. Even when you talk to regular people about AI Safety, if they get it, they tend to integrate AI Safety into their pre-existing worldview which tends to be partisan. When those individuals then share their views on AI Safety (in person or on social media), they tend to bring along all the partisan trappings of their wider worldview to the message.
I suppose the root reason I am skeptical that AI Safety can remain non-partisan is because, in a sense, once you share the AI Safety message, you can’t really control how the person you are talking to processes and integrates that information into their worldview, and our society is set up in such a way as to naturally pull important issues into existing partisan identities.
hmmm, I think that the Clinton/Dole race of 1996 is instructive here, because Clinton set himself the task of moving so far to the right, that Dole had little platform left to run on (at least as FdB tells it). We might end up in the scenario where AI safety becomes so popular that politicians are trying to out-do each other on who can be “tough on AI”.
But generally I think it’s best to start from the OPs perspective, of avoiding partinization at all.
Why would government regulation of private industry to reduce negative externalities not have an innate left partisan valence in America? Even before Trump and right populism, this would be seen as a textbook left policy
The modern Republican party supports all kinds of government regulation of private industry. The most obvious immediate example is tariffs.
Tariffs are actually taxes (in practice passed down to consumers) not regulations to reduce negative externalities
Does the distinction matter, in this context?
The way it’s officially presented to the voters by Trump is that foreign countries not American private companies pay this tax. In practice American consumers pay. I struggle to understand how this all might relate to the regulation of the American private industry, especially in this context.
If you believe “the modern Republican party supports all kinds of government regulation of [US] private industry”, please present an actual example! Shouldn’t there be plenty of examples if “all kinds” are true? (Reminds me of https://www.lesswrong.com/posts/pWc6ZA6mcjXKvKwPd/discourse-norms-moderators-must-not-bully/comment/umELK75yshpZmug2D)
Some examples include (in no particular order):
Non-tariff import restrictions (e.g., you can’t buy a Chinese electric car)
Various forms of social media and child safety regulation
Restrictions on Chinese ownership of and investment in U.S. assets
E-Verify
The crusade against institutional-investor ownership of single-family homes
Bans on abortion and gender transition, in particular ones that attempt to expose providers to civil and criminal liability
Some MAHA stuff, like restrictions on food dyes
Anti-debanking
Anti-ESG and anti-corporate-DEI
“Regulation of private industry” is admittedly not quite a fully black-and-white concept and I suspect you’ll argue that some of those shouldn’t count, though I can’t guess which ones. (Although the tariffs I think really should; if politicians can impose them and simply deny that this is government regulation, then why not do the same with AI?) Also, most of these are not universally supported within the Republican Party (if for no other reason than because it contains some actual libertarians), some Trump has gone back and forth on, and some are supported by at least some Democrats (populism makes strange bedfellows, as does the increasingly strange role of organized labor). But the overall picture seems clear enough.
(Comment link seems to be broken.)
Because you can frame this in many other ways, e.g. “Defending our homes and families from an alien force”. Even on a contemporary-meme more right-coded things are already directly associated with it, e.g. “Butlerian Jihad”.
You say even before but you just mean before. I think it’s agreed and obvious that there’s been a pretty big shift since trump’s populism toward “government should do things I like” which probably includes regulation.
Border control is a straightforward example. The Right Wing position is that totally opening the labor market introduces negative externalities that aren’t factored in when businesses calculate personnel costs. To that ends, the Koch wing of the party has been steadily losing influence due to its refusal to back down on this issue.
AI, at least under the Safety narrative, is similar. Corporations are bringing in a threat to our public commons in search of short-term profit, and our collective birthright supersedes their right to do so.
Nope, it’s as far from straightforward as it could possibly be. Border control issue is not driven by labor factors at all, and the Republican party doesn’t even pretend this is the reason, as everybody knows that Latin American migrants actually work in the sectors like agriculture where US citizens don’t want to.
It’s officially presented as “law and order” issue but I would argue it’s actually driven by xenophobic sentiments of the Red voters first and foremost
Perhaps “pre-existing” would be a better word choice than “innate”, but I did write a whole paragraph about why Red is inclined to oppose AI (opposition to social upheaval, suspicion of Big Tech, etc).
I chuckled, then thought for a few seconds, and that’s actually not so bad an idea.
I think someone trying that isn’t a bad idea, but Eliezer might not have the stamina. We should figure out who would make a good cowboy to be the face of AI safety to rural America.
Please explain vice signaling. (That is something that the rationalist materials ignore, but it exists in real world.)
Agree, the immediate advice “please don’t bring up ICE protests for no reason” is right.
My counterpoint is like, you should still do deep work with both parties/tribes, and progress is typically getting deep buy-in from one side plus cooperation from the other. I’m hoping for a better framework for when/how to be partisan. Given that I think the left is also at risk of “it’s just a stochastic parrot”ism, I think there’s risk of failing to persuade the left, too.
This seems like shades of Politics Is The Mind-Killer.
While theoretically desirable, I’d worry that what actually ends up happening if we can avoid partisanization is that AI safety ends up becoming the target of both extremes of the political spectrum, the way “centrists” are. There’s already some of that happening with people on Twitter calling Effective Altruism a “woke globalist cult”, and people on Bluesky/Reddit calling Effective Altruism a “billionaire techbro grift cult”. Though, that’s EA rather than AI safety per se.
I’m biased because I’m left-leaning, but I see making common cause with the lefty anti-AI crowd to likely be more effective at actually slowing AI development than convincing the accelerate AI boosters to change their minds. Sure, there’s probably some skeptics in the right, but most of the skeptics like Ed Zitron, Cory Doctorow, and Gary Marcus are much more left-coded.
I note that most of the AI doomers like Yudkowsky are on Twitter rather than Bluesky, which means AI safety is already relatively right-coded anyway.
The way I see it, there’s currently three main factions in the AI discourse right now. We here on LW are mostly AI doomers. Adjacent to us and at the same time semi-opposed are the AI boosters. Opposed to both are the AI skeptics aka the anti-AIs. Doomers and Boosters are already right-coded, while Skeptics are clearly left-coded.
Reality is, the boosters are the ones who are least likely to slow or pause AI. The skeptics are often incoherent and emotional in that they think AI is simultaneously useless and a threat to their jobs, but at least they oppose rather than support acceleration.
People calling for AI safety to court the right, strike me as being oddly unaware of how already right-coded AI safety is, at least in the terminally online universe of social media that I see (which admittedly is more leftish than the average Less Wronger).
So, to me, AI safety is already partisanized, and not in a good way.
A lot of the skeptics, by the way, to caricaturize for a moment, see AI safety as a grift designed to create hype for techbros selling useless generative “AI” before the bubble bursts. That’s how far away that pole of the discourse is from Less Wrong.
What partisan split would you predict among those with AI safety careers, e.g. at METR?
I don’t actually know a lot of the people working in AI safety or at METR personally, so I’m hesitant to make a prediction. I do know that past EA Surveys have usually shown a large plurality of centre-left EAs. People working in STEM tend to be a bit more right-wing than average, but having a university degree also skews liberal. Less Wrong has some vocal right-leaners like Richard Ngo, but some like habryka seem to lean slightly left?
If I had to throw numbers out there I’d guess something like 55% right, 45% left with many people actually being mostly nonpartisan (track the truth where ever it goes) with weak leanings.
People’s self-reported positions would also be different from perceived positions. In my previous comment, I’m mostly talking about public perception, which tends to lack nuance.
Over the last week, I’ve seen prominent representatives of both “right” and “left” promote bad takes on AI safety, so I’d suggest that this is already fairly bipartisan, though not in a good way.
A common starting point seems to be that no one trusts AI further than we can throw it but then commenters go in quite different directions from that. And the directions overwhelming reflect pre-existing political biases.
Compare “AI is already bad for the world and getting worse, we need to ban it via international treaty” with “No we need to go full steam ahead because if we don‘t China will”.
Or compare “This is marketing hype to keep the AI bubble going” with “These firms are now pushing for a detente because the costs of the arms race are destroying their profitability”
In my opinion this post is problematic for the following reasons.
1. The post doesn’t practice what it preaches: The post urges AI safety activists to not develop a partisan public image, yet it argues overwhelmingly with ‘mistakes’ by the left leaning side. We first have Turner and ICE. Turner states the ICE incidents as the starting point of his activism. Should he just omit this information although it is vital for understanding his story? Then Ngo. Here, the side that denied him the job because of his tweets was criticised. You could similarly argue that his tweets were partisan and that this was the original mistake. Aella was mentioned to. Here the claim was purely speculative and no source was given that actually demonstrated that the image was perceived in a controversial way.
Again, only the left-coding is identified as the problem. The article ends with the following suggestion:
This sounds quite red-coded for me.
Overall, for me the article is making less of a point about ‘AI safety should not be partisan’ and more about ‘left leaning voices in the AI safety discourse should self censor’.
2. The article doesn’t clearly separate facts from personal opinions:
What exactly makes the accusations emotional and logically unexamined?
Cannot possibly. Why not?
3. It misses the opportunity to list counterarguments: I believe that by keeping everything neutral, you might lose most of your credibility and actually won’t land well with any political spectrum. I think, we should want to sound like normal people that can be related with in their concerns regarding AI safety. Normal people seldom only have one singular political concern. As an example, the ICE part of Alex Turners article was something important for me and it kept me more engaged with the article. There might very well be different posts regarding AI safety that cater more to a different political spectrum.
Another point:
Well, but what if this is true and both are in fact bound together? Should you just ignore the connection?
As a last point, I want to question the relevance of the proposed analogy between the Culture War and religious conflict for the core claim of the post. Is this analogy necessary/helpful for the post? Especially the point about the duration is unclear for me. Why should an analogy to events in medieval times tell us anything about the relevant time scales? Social networks could dramatically influence the speed of the conflict.
It actually were not any leftists nor liberals but the alt-right/right populists who promoted the vaccine skepticism in the “Red tribe” out of anti-expert sentiments. Right populists, who dominate the modern GOP, define themselves as the enemies of the educated elite. I believe Dr. Fauci could do nothing in 2020 in order not to become an enemy of the right populists (hence the need for a presidential pardon in 2024), even though he had a privilege of being an old white man (as opposed to Coxton and especially the METR staff, many of whom were unfortunate to be born female)