AI Safety Can’t Afford to Pick a Side on Consciousness
TL;DR—More of the public than ever are talking about AI consciousness and they won’t wait for research to split into their pro- and anti- consciousness camps. The pro-consciousness camp could threaten the control paradigm by viewing monitoring and training as violations, while the anti-consciousness camp could over-invest in control paradigms and miss opportunities for collaboration. Trying to sway this debate is too costly—instead, AI safety advocates should focus on reconciling the blocs and creating compromises.
An edit for clarity: The title urging not to “pick a side” is meant to say that AI safety advocates should not align themselves with a political bloc, not to say that advocates should not take stances on propositional claims. Advocates should never misrepresent evidence. Staying out of consciousness politics matters most while evidence is unclear but, even if evidence becomes clearer on consciousness, the top priority should still be collaborating with disparate groups towards AI safety. Thanks to @Vladimir_Nesov for pushing me on this.
AI consciousness breaks containment
The AI consciousness debate has not generated a ton of traction on LessWrong. However, it seems that the topic is becoming popular in non-specialist circles.
A lot of this is spurred by the explosion of the AI pain axis study over Twitter. There are millions of views on the tweets, sure, but I also have people in my life who have never shown interest in AI asking me about my take on it because they saw a meme on gaytwit.
We also see that during the Hugging Face news, searches for “AI rights” exploded compared to other AI topics including extinction and hacking. While “AI consciousness” did not increase, rights are generally used as a proxy for consciousness and welfare—see the label of “animal rights.”
Google Trends data: “Hugging Face” in orange as a control, “AI rights” in blue. Other related terms lke “AI hack,” “AI consciousness,” and “X Risk” below. “Animal rights” also used as a control.
A few days ago, a limited poll found that 46% of Gen-Z believe that AI is conscious.
As I was writing this, a New York Times alert informed me of an article on AI consciousness.
The topic is divisive in both academic and public circles. In academia, especially, it continues to evolve. If you have the time, I encourage you to directly read the outputs from the Eleos Institute and Google Deepmind, as well as some of the summaries on LW or recaps on Digital Minds. For views more critical of AI consciousness, see Lennon 2026 and Chad Woodford’s extensive Substack series.
We currently lack data on public views of AI consciousness, but I hope that this gap can be filled soon. Data has come out about Americans’ fears of AI power, but we don’t yet have the crosstabs I’d want to see here to draw conclusions on consciousness.
Promising stuff! Source: Echelon Insights.
Emergent views on AI consciousness
The goal of this post is not to argue whether AIs are conscious or not or even work out the implications of AI consciousness.[1] It is to argue how popular disbelief or belief in the consciousness of AI will affect the political project of AI safety. The bottleneck, after all, is political will.
I do not know how the debate will play out in public nor do I know if one position will ultimately dominate.[2] There seem to be many different incentives for the beliefs of everyday people.[3] I would estimate that a majority of people with opinions on AI consciousness will believe that AI is conscious by 2030. However, only a third of these believers actually change their actions. What I am confident in is that public opinion is unlikely to consult the complex scientific research on AI consciousness and even less likely to wait for it.
What follows are my predictions of trends in two blocs of public opinion. I have set aside the people who have beliefs but do not act on them or have no beliefs on the topic. I acknowledge this is probably a majority of people. Still, engaged blocs matter, and I hope this can allow us to best plan how to continue to support AI safety in the light of opinions on consciousness.
AI-is-conscious bloc
Some who believe AI is conscious will not change their behaviors because of it (as some people who believe that animals are conscious but do not change their behavior). This category, then, speaks to those who believe AI is conscious and who will change their behavior.
Trends I expect to see in this bloc:
Greater sympathy for distress expressed by AIs. This could lead to wanting to minimize AI pain (or “pain”) and generally being accommodating to what AI agents express as want. This could make this bloc both more susceptible to scheming and a better deal-making partner at the same time.
Greater anthropomorphization of AIs. Our most available model for consciousness is humans, so it will be projected onto AIs. Practically, I could see this leading to more social attachment to the AIs in their life. It will also lead to assumptions that AIs will act like humans in a given scenario.
Religiosity concerns. The NYT article starts to touch on this, but consciousness is understood largely through religion for billions of people. It will become a topic in those who believe that AI is conscious.
AI-is-not-conscious bloc
This position is mostly the status quo, but I expect it to develop as a consciousness bloc emerges. Like the above, this should only include those who believe that AI is not conscious and change their actions because of it. I use scare quotes liberally here to reflect the views of the bloc.
Trends I expect to see in this bloc:
Greater openness to extreme “treatment” of AIs, both in training and deployment. This bloc will not be concerned with RL or the destruction of models except if it proves to be unsafe for other reasons. There are also likely to be fewer qualms about the kinds of roles AI can be given, such as in warfare or menial labor.
Greater investment in the control paradigm of AI. Their approach to safety will be based more on the idea that the capacities and powers of AIs should be controlled by humans and that bargaining is one strategy among many. They are more likely to “deceive” AI.
How the AI safety movement should respond
I do not think that the AI safety movement should expend its limited political capital on convincing people that AI is conscious or not conscious. It will likely continue to be scientifically unclear for a while and people seem to have really strong priors on what consciousness is.
Instead, advocacy for AI safety should continue with an assumption that there are different beliefs about AI consciousness.
This will be difficult because it is likely that AI consciousness advocates will reject much of the current paradigms of control in AI safety. When anthropomorphizing AI, current safety procedures like mechanistic interpretability can be spun into incredible evils. It will not work to dismiss these risks out of hand, but I think the AI consciousness bloc can be convinced that certain inconveniences are necessary for safety. For example, we humans accept airport security because it’s annoying but necessary. Could the same not be said for mechanistic interpretability or CoT monitoring? This argument is much more likely to work than citing studies and calling people uninformed. The political nuance will be critical, especially if religiosity does get involved.
On the other hand, we can see the AI consciousness advocates finding new and creative ways to create alignment with AIs. For example, AIs may be more willing to be honest or make deals with people who assign them more agency—in the pain axis study, “denial of consciousness” by the user seemed to cause the AI to represent pain to itself. These advocates also are critical to ensuring that welfare considerations of AIs are considered even if there is just a risk of consciousness.
There is also the need for more research. Personally, I would like to see research into how views of AI consciousness affect views on a few topics I don’t think obviously map to either position:
X-risk
Pacing the frontier
Interpretability
We should treat the emergence of these blocs as a likely given. Good policy will not be in dominating one bloc or the other, but in reconciling them. This, after all, fits with what the research tells us on AI consciousness: there is no clear answer yet.
- ^
Additional reading if you are interested in this topic (aside from what I’ve already linked):
“Taking AI Welfare Seriously”—Long and Sebo et al 2024
“Where is the Mind? Persona Vectors and LLM Individuation”—Beckmann and Butlin 2026
“What does ChatGPT want? An interpretationist guide”—Goldstein and Lederman 2026
“Artificial symbiotic intelligence: Agents, AGI and the orchestration of many minds”—Bratton et al 2026
- ^
One could be very tempted to say that the “anti-AI” position will dominate in the US context, since the US holds very negative views more broadly. First, I am not sure whether it is more “anti-AI” to believe in consciousness or not. Moreover, it is not clear whether those who dislike AI also believe that it is not conscious. Negative views of AI and views of general AI risk tend to correlate, but views of AI x-risk correlate more with perceptions of AI capacity, not of dislike.
- ^
If you are afraid an AI will take your job, it is unclear what you are incentivized to believe in terms of consciousness. On the one hand, granting consciousness to something that can harm you risks empathizing with it. On the other hand, it may feel more reassuring to be replaced by a conscious machine than an automaton.
This framing permits the possibility that it could be desirable for “AI safety movement” to “expend its political capital” in order to start convincing people on claims it doesn’t believe to be correct. Not consciousness of AI perhaps, but some other claims. This general possiblity about persuasion seems more of a crux, making all the details of specifically the AI consciousness debate irrelevant (as long as the fact of it remains unclear).
I will go so far as to say it will be desirable. This post on political will explains my views on this pretty well.
For example, it is worth expending political capital to convince people (and legislators) that AI can be an existential threat. It is worth expending political capital to educate people on core concepts.
I actually think the political will calculus may end up being more important in the next few years than research. Orgs like CG can just keep funding research, but the levers of power are limited by things more complicated than money.
My point is that if you are not going to start persuading people about claims that you don’t know to be true, as is the case for AI consciousness (or absence of AI consciousness), then it doesn’t matter if you are willing to “expend political capital” in this particular case, you wouldn’t be doing it regardless. So the fact that you are discussing these considerations (while simultaneously saying that the truth of the claim is unclear) suggests that you are operating under the premise that if the political capital situation was different, persuading people about claims that you don’t know to be true would be on the table.
It’s unclear to me what the “it” in this sentence refers to. For “AI safety movement” to “expend its political capital” in order to start convincing people on claims it doesn’t believe to be correct? (That would be the interpretation if this is answering my first sentence, “the possibility that it could be desirable for …”.) Hopefully a misunderstanding, but just to be clear that it’s what my first sentence actually flagged (if you are reading something else into it).
Oh, I misunderstood your original comment! We agree. Part of the reason I think we shouldn’t use polcap to convince people one way or another on consciousness is exactly because we don’t know (I am somewhat convinced that this uncertainty is structural). That said, the truth of that “not knowing” doesn’t really need to be promoted.
(Btw, the “it” just refers to expending political capital in general, which I think is good.)
The AI safety movement should probably not expend political capital on advancing claims it doesn’t think are correct. But the inverse is not true here—AI safety should not necessarily expend political capital on attacking all claims that are incorrect. This is the element of strategy—there’s only so much political capital and some outcomes are more important than others.
(For example, I am more concerned with persuading someone to support tax reform than persuading them to be an atheist or persuading them that children shouldn’t play football, though I believe that all of these positions are correct.)
It’s my sense, though, that there will not be a “correctness” to the AI consciousness debate. There will be people who have the impulse to try and communicate this “correct uncertainty” to people, but I think this is a waste of resources (partially because it is hard to explain, partially because people have priors.)
There is probably some broader political strategy/utilitarianism argument here about whether truth is a prerequisite to politics or whatnot, but I am not qualified to dig into that claim.
The reason we shouldn’t pick a side is that we don’t know which side is correct. That, on its own, should be quite sufficient to argue that we shouldn’t pick a side. Raising the relevance of some other reasons not to pick a side suggests that it’s not quite sufficient. That is the issue I’m attempting to point out.
You now seem to be saying that the political cost means that “AI safety movement” (whatever that is) shouldn’t promote the claim that “We don’t know whether AI is conscious”. This doesn’t at all fit with the title of the post (“AI Safety Can’t Afford to Pick a Side on Consciousness”), or most of its content. If persuasion about things-not-known-to-be-true is off the table, then this post, being focused on AI consciousness specifically, and discussing political expediency, would only make sense in this strange interpretation of saying that the concerted and costly promotion of uncertainty about AI consciousness is undesirable. I don’t think this is what the post is saying.
The debate may fail to find correctness, and the claim itself may be too confused to centrally admit correctness, but promoting claims (as known) while their correctness is unclear is deception. When correctness is unclear, promoting claims of correctness being unclear is honest, though I agree it might not be something worth focusing too much on. It’s also not worth framing a post like this the way it was framed, in that case, or giving it a title about “picking a side”. In this last comment, you seem to be indirectly implying that your intended meaning in the title of the post was that the side of “it’s unclear” shouldn’t be picked...
I think I’ve been unclear, especially in these follow up comments, and that’s my bad. I provide reasons aside from “I don’t know” to hedge against the possibility that we may, in fact, better learn the “correctness” of AI consciousness. My argument is that even if we learn that one political bloc is more correct than the other on consciousness, it still doesn’t mean that AI safety should align with it.
(By “align” or “pick a side,” I mean a political side—an acting bloc organized around a claim—not a side of a truth claim or a proposition. These are different things. The “side” of safety should be independent from picking “AI is conscious,” “it’s unclear,” etc.)
This is a claim of prioritization of political effort and strategy. Safety advocates shouldn’t promote claims they don’t believe but I still think advocates can work and engage with both blocs without ever having to do so. The coalitional dynamics will get gnarly with the blocs, but that’s all the more reason not to align with one and keep the focus on AI safety.
I don’t care two hoots whether the AI is “conscious” for any arbitrary one of the dozen-or-so shades of meaning that word has. I do care whether it responds in different ways if we treat it well, or not — i.e whether us choosing to act as if it has moral weight, or not, makes any difference to its behavior, and if so, which option makes us safer.
This is basically the interpretationist stance of consciousness, explored by Goldstein and Lederman 2026 in “What does ChatGPT want? An interpretationist guide.” I’ve also read similar utilitarian versions of your statement.
I do not think that much of the public shares this belief. It seems to matter to people whether an AI is “conscious” or “real” or “sentient.” We will see if these beliefs change behaviors—that’s what I’m keeping an eye on at the moment. If it does change behaviors and political beliefs, it doesn’t matter what is “true” or “cared about” by researchers, it’s a political reality we will have to adapt to.
Personally, I am interested in consciousness research. I do not presume my interest means global importance. That’s part of why this post doesn’t actually go into consciousness research, it just takes that it will probably not resolve and that the public will catch on to debates.
I’d call my approach to metaethics a contractarian one (along the lines of David Hume), or more accurately a sociology of morality one. And you’re quite correct, many people do think that “ought we to give X moral rights?” has something to do with “is X conscious?”, for some interpretation or other of what the word “conscious” means.
Personally, I’m a moral anti-realist and constructivist, i.e. think that society gets to pick and chose rules like that. But we don’t get to pick and choose on whether granting AI moral rights makes us safer, or less safe — that’s an objective question, and one the survival of humanity may well depend on. Personally, my best guess is safer: if you want someone to treat you as having moral rights, normally you acknowledge that they do: that’s how human societies work, and anything familiar with human values, as LLMs are, will also be familiar with that assumption.
People who care about AIs should support stopping creation of current variants of them in current circumstances. Which would also help safety.
Yes and no. I think it depends if the pro-consciousness block is pro- or anti-”natalist” for AIs. I can see any of the following playing out:
Creating variants involves RL and other “harmful” behaviors, therefore we should not create variants.
Creating variants creates more conscious AIs who experience negative welfare, therefore we should not create variants.
Creating variants creates more conscious AIs who experience positive welfare, therefore we should create variants.
AIs are conscious so we should allow them to do what they want insofar as it does not harm others. This includes allowing them to create variants.
AIs have a “right” to reproduce, i.e. create variance.
Note that all of these assume that ideologies are rational and logical, which is probably a faulty assumption.
Maybe more likely scenarios that are a little less logical:
Pro-consciousness folks protest at the labs and the labs make the concession of having less variants to appease them because it doesn’t cost the labs too much. This leads to less variants.
Pro-consciousness folks get attached to specific persona-instances or slices of AI because they believe them to be conscious. This leads to more variants because of 1. preservation instincts and 2. economic incentives.
Pro-consciousness folks have no idea what a variant even is, so they don’t focus on it. This leads to ? variants.
Political predictions of predilections is really really hard, which is why I call for more research not on consciousness, but instead on the political implications of beliefs in consciousness. Anyone who is certain that beliefs will lead to a certain kind of AI safety politics is likely wrong.
I’m just saying I am anti-natalist for AIs here and there are things to say in support of it. Like, I think people would be more on the anti-natalist side if it was about humans in place of AIs.
And yes, it doesn’t mean people will think like that—it’s not like there is a consensus on anti-natalism conditions in cases of animals or even humans.