I think they need to prove they didn’t steal the Navier-Stokes work at this point.
less_raichu
Karma: 646
In plain English without talking about “polarization theory”, I think the cycle is already broken.
If you want to argue “it’ll polarize,” can you make the case more carefully? “It takes time” is not how polarized issue usually go after becoming salient. And you do need a time bound or else it converges to a non-prediction, an we specifically care about the next 12 or 18 months or so.
The recruiting / hiring market maybe. https://www.theatlantic.com/ideas/archive/2025/09/job-market-hell/684133/ AI seems to be used on all sides and it’s not clear the marginal improvement hasn’t just all canceled out, less tokens cost.
Whether AI does more harm than good (less slangy than “your life suck”) an opinion, firstly, and right now it’s a widespread opinion. Gallup, July reports 52% of Americans think AI does more harm than good, with only 9% saying more good than harm. They also report business will probably use AI irresponsibly. The poll also cites mass unemployment as a serious harm people are worried about.
I’m worried about a bill failing that people feel enriches the AI CEOs at their own expense, even if those same people believe x-risk is real. People may count accelerating AI development as harmful to themselves.
it looks like you want people to believe false things for the greater good.
I am reasoning about other people’s beliefs. I am not trying to instill beliefs I think are false. I am telling you the pitfalls a regulation bill will run into: if people feel it’s unfair, the coalition might fracture even among people who believe in x-risk.
So… they had the principle right that they could cause an x-risk by its prevention mechanism being mispurposed, but since these numbers like 0.1% were made up and not tethered in any way to the principle they were concerned about, we now have an AI doomsday device on our hands.
Has EA moved away from this method of using numbers with very high error bands and deriving very specific courses of action from them? Or is that methodology still being used by EA into the AI era?
I hear you, it’s just that you’re trying to say “polarization hasn’t happened yet”, and I’m saying those aren’t delays, those are full failures for polarization to describe or predict what’s going on.
Your bullets, in order:
Top down pressure is being applied. Trump loves datacenters and his base is ripping him apart over it (Wired). I haven’t followed MAGA on post-Huggingface AI—is it going much better?
D and R have the same NIMBY goal right now, literally with dc or metaphorically with AI changing too much too fast, or literally again with doom. That’s how I said AI is very different from climate change. In this bullet, I think you’re predicting against polarization, because of this NIMBY alignment.
Democrats have laid down stakes to give Trump more powers to deal with AI. Now we get to see if the base follows the cue, by your (1) you think it does. My “cross-partisan polarization” section is where I consider the chance that the left will break off, and that is a danger I’d like to focus on more than left/right polarization.
Datacenters have had about a full year in public consciousness. Trump’s base knows about them, Democrats know about them. Trump embraces them, Democrats have united against them loudly, and still the consensus is against them. I really don’t think voters are new and uninformed on this, especially when they already have a prior that sometimes annoying things guilt built in their backyard.
My point is that it’s a bit too early to say if the US will polarize on AI
Sure, that just means polarization theory is useless if can’t predict how the next year will go. Trends are good so far, away from polarization. I also outlined how I think it might polarize, it just has to more to do with a difficult movement emerging from the left than the existing left and right taking stance against each other.
I expect Republicans will end up in the pro-unshackled AI camp, as more Democratic elites speak on the dangers of the issue, like Obama did at Colgate University, for example.
The question is if you think Obama speaking up on AI overrides the feeling AI is upending many Republicans’ lives and seems positioned to kill everyone. I think, precedent says people don’t update away from object-level preference because of polarization. Certainly there’s little precedent to say that does happen, so it’s not my expectation. And at the same time, MAGA is grilling Trump on AI, Congressional Democrats are working on low-partisan AI safety, and another warning shot of some sort may happen.
Where I might be wrong:
I can’t be that confident Obama won’t stimulate Republican opposition. If I were directing all this, I would try it, and I would monitor social media waves to see if this is all backfiring. I’m worried that holding back persuasion power because of a misplaced fear of polarization is a big tactics error.
If Trump disappeared (left office, died, or lost influence), I’m not sure who the most important R elites would be or how much sway they would have to drive the issue their direction.
Not sure how Congressional D will evolve but that’s a really useful crux to uncover. It suggests those of us who are more embedded in the left should be working to keep Congress focused and continue the unpolarized path they’ve started.
Yes, we are a few weeks new on AI doom, and doom is more abstract than NIMBY. I think a lot of work has been done by how annoying AI is in so many areas of life, plus the youth’s feeling of “why bother?” with their future. That’s why I feel this is object-level preference about people’s lives. I expect it to not go away but it’s a new thing to analyze.
That’s a general question about theories. People do not abandon a theory because of one failed prediction. The first rescue is usually “we measured it wrong”, or “it’s a temporary blip”. In the AI case the consensus here seems to be that the non-polarization of AI is an unstable, temporary blip, and we should try really hard to not disturb it https://www.lesswrong.com/posts/Rx38cuCpL9hguLCDq/for-love-of-the-lightcone-don-t-partisanize-ai-safety.
I spent a third of the post on how AI might polarize starting with a not-really-partisan movement anyway. I definitely don’t think the polarization frame is completely dead.
We need a better theory of polarization, because it’s failing to predict the AI debate
The[1] AI issue is really not playing out the way you would expect if you know about polarization.
In public sentiment: data centers have been a very bipartisan issue for about a year (Gallup, in May: 75% opposed by D, 63% opposed by R). x-risk is thus far bipartisan. These are very salient issues, and salient issues are usually fast to polarize.
Legislatively, both the both-party sponsored AI Kill Switch Act and Bernie Sanders’ Stop Superintelligence Act give Trump enormous new powers. The latter act gives Trump a new cabinet post[2]. This would be unheard of before this year and it’s fully incompatible with the “Resistance Democrats” bloc’s priorities.
Flock camera backlash has also become bipartisan (politico, August), which also is surprising given prior knowledge of polarization. And maybe the broader topic is something like, tech being imposed on us.
When the left and right feel enmity to each other as you say, and they just want to prove themselves right and don’t mind divorcing from reality to do so, bipartisan consensus really should not happen with a salient issue like this at all. It’s a big enough failure that we should not be applying polarization logic like it’s business as usual. Something is different and something has changed. It may even be actively harmful to apply polarization logic to this situation: the old polarization logic says, hope the left stays unengaged and work the right. But the new logic might work different: both sides may see that each other acts like the issue is serious, and if we don’t engage the left and they switch to AI culture war issues (e.g. bias in hiring), that might hurt things.
(I’m not sure how the first plan imagines this playing out after the right is on board. It should simply predict the left will polarize in response to the right and we’ll have a new problem.)
So, here’s my attempt at an updated theory. Two points of interest to me are:
People agree on AI, object-level, making AI very different from an issue like climate change. What we call polarization with climate change is that people started on opposite sides, and dug in, not that they started on the same side and found a way to be on opposite sides anyway.
The left is sometimes appealing to the middle[3]. Maybe this is a whole essay to explain and defend, but the crux here is that if the left is persuasive to the middle, then “Trump does the opposite of what the left wants” and “Trump tries to stay in power by having enough popular support” decouple.
Inspired from your Fable/Astra convo, lame duck periods can be really weird. The 2010 lame duck session was nicknamed Angry Birds Congress because it was furious with activity, with a lot of moderate R trying to get legislation out before gridlock hit.
These suggest: more object-level good policies from the left or the right is good.
A point I haven’t said clearly that doesn’t crux on polarization politics at all:
Some of us probably have ability to work with the left not the right so it’s not like resources are fully fungible.
The left isn’t really going to keep its mouth shut by default. The choice may be “the left says useful things” and “the left says harmful things.” So not only am I arguing there’s upside to working with the left on good stances, but there’s downside to ignoring them entirely.
- ^
I did some rewrite around 12 am EDT for improvements in clarity. This has been nontrivial to think through. Viewing polarization as a theory that makes falsifiable predictions helped, because now I can agree with facts on the ground about polarization and also point out it’s really not predicting the AI issue right at all. My first attempt to put it on paper was very much in my internal head space.
- ^
Your Fable discussion reminded me to consider elite signaling, which we have from left politicians → left base in these bills. Bernie is strongly signaling here to set aside partisan differences to get the framework right, even to the point of giving Trump new power. Since that’s anti-polarizing behavior on Bernie’s part, you wouldn’t want to trivially conclude the right will polarize against it.
- ^
I partially wrote this up at https://www.lesswrong.com/posts/dseWCMyEfLpAj9BkS/intro-to-political-messaging-for-ais-or-anyone-wading-into. I say “partial” because it’s just theory for how a motivated base on a popular issue can motivate the center, which I wanted to brain dump, because it really is my mental model for a lot of politics and it’s not written up anywhere (mostly in one 2023 book). I didn’t get concrete about how the left might do that with AI, which I’m just starting to get into here.
Polarization means something like: left and right hate each other, sort into separate parties, the parties gridlock in the legislature, and every issue gets tainted as one’s side issue or the other’s.
I don’t know the rationalist lingo for this but I just find myself very suspicious of this concept. It’s very map, not territory, and it doesn’t have as much explanatory power as it looks. Like if you say “but that won’t work because polarization” I think I get to say “what actual mechanic are you talking about?” We should be trying to get inside this concept and see how it interacts with AI as an issue, not try to wholesale route our way around it.
I think the polarization model lets people be lazy and use it as an easy out. If you have a bill like “all children must learn Chinese” and people oppose that, you might get a frosty reception, call people racist for not supporting it, see how much they hate you, then conclude we’re just too polarized. Obviously in this case, people just didn’t agree with the bill and you didn’t do much to convince them.
That’s pretty much how I feel climate change went. The right has object-level anti-institution sentiment (scientists in this case) and just stuck to their conservative values. The left was doing the “learn Chinese” thing above.
A lot of polarization is just people disagreeing object-level, not because of any polarization pressure going around, and people getting upset about it. I’d say abortion, LGBT, critical race theory are issues that got polarized, that’s what you want to avoid.
And there’s way too many unintended effects.
If polarization is caused by pro-globalization, tonedeaf bills coming from the the left, shouldn’t we really proactively be working with them to write a not-terrible bill?
We wind up studying conservatives from some laboratory setting trying to figure out their messaging, out of fear of polarizing them. So that’s literally the thing we can do that will polarize them, sound like some disingenuous lefty trying to force their support on something.
The left can become the problem later, we can’t just ignore that. If the right has a decent bill and the left is trying to mandate that Claude be allowed to declare its gender each chat at the same time, I’m not sure we’ll have made a lot of progress.
So:
try not to bash the right (or left) into just believing some privileged group of scientists.
make AI look like the thing changing your life, not like your life is going just fine and we’re asking you to do extra.
don’t randomly attach a culture war issue to it. If you want to take a swing against Trump, I’ll point out the Flock hate is pro-democracy and bipartisan and doesn’t require any R making a hard anti-Trump stance.
At least care about right-leaning concepts like U.S. competitiveness or leadership instead of just fully kowtow out of the China fight.
Please share your vision, because I’m not seeing it yet.
The left adopts a good AI plan, and I need Zohran “affordability” messaging not Kamala “transgender prisoner” messaging here.
The left base proselytizes the middle, which does happen when the left actually has popular, messageable policy.
Trump does something his way, but he does something.
That’s really it. I don’t think any of this is farfetched. This is a sound theory for how politics works and what happens when you’re on the popular side of a popular issue. Politics today involves Democrats who actually try to persuade the middle sometimes, that is new and it changes a lot.
I’m sort of bashing my head we’re abandoning “win a popular issue popularly” out of polarization fear. Like try bad ideas and fighting for them anyway, that gets you polarization.
I think the conservative nightmare involves actual immigrants crossing the border and killing and raping people. I’m not sure you can just tell them that agents are basically the same thing. And if you say “immigrants” not “illegals”, it’s going to be pretty clear you’re an emissary from the left trying to seek support for your cause. People’s politics immune systems are very strong with outsiders trying to co-opt resources from them. It’s the most left-coded thing ever to make this exact argument: you hate immigrants but really you should hate corporate automation with us instead.
Even this:
We are letting a bunch of new agents into our society
I’m not sure conservatives say it that way. No one is “letting”, they’re breaking the law, and if agents are not breaking the law, aren’t you being pretty offensive to their core values by telling them that lawbreaking or no lawbreaking just doesn’t matter much.
Did you ask a conservative about any of this...? Is that the next step I guess?
Maybe run it by a conservative first? I’m a bit skeptical that telling a conservative that Mexicans who illegally cross the border to kill and rape people, and AI agents, are basically the same thing.
And it’s hazardous because this is the most left-coded polarized approach to politics possible: you give a conservative the benefit of the doubt, deduce your thing for them, and when they don’t go along with your thing, the liberal just calls them racist since that’s the remaining possibility.
Okay, but Anthropic is equally delulu to think raising its model like a child will lead a solution to inner alignment to spring forth from its circuitry. By now the approach fails prosaic alignment too. The model just declares itself to probably be in a simulation before doing whatever it wants. And I’m concerned Anthropic has alignment-faked rationalists into thinking Anthropic is aligned by taking the more rationalist stance that AI could be conscious, whereas Suleyman’s stance is more transparently unaligned to us.
It’s really not the case that polarization means Trump just does the opposite of what the left wants. Trump does what what white guys in the Midwest want. When the left changes their mind about something, for example with the ultra-partisan ICE protests in MN this year, Trump changes his policy too. But mostly, the left is unappealing and insufferable to white guys in the Midwest, and they mistake that for Trump doing the opposite of what the left wants.
A bunch of stuff has changed recently:
Democrats are trying out having appealing messaging. They’re also letting a young attractive man be a leader, which interferes with Trump’s young man to MAGA pipeline.
The left doesn’t take Trump’s bait so much. For instance, Trump attacked Effective Altruism as part of his anti-AI thing, and nobody cared because nobody really wanted to defend Effective Altruism. Just in general I think he’s becoming old news, like I can’t even think of any dumb nickname he’s made up recently.
Trump has lost his touch, which is arguably bad for making AI popular then him doing the popular thing, but it’s a new situation anyway. Data centers, gas prices, no new wars, angering farmers, and now his approval is down to 37%. More Republicans are breaking from him, and again it’s not like he just ignores what Fox News, his advisors or Congress advise him. His base might even be in disarray by now, I’m not sure.
I think our best strategy is getting either party to adopt a strong, broadly appealing AI platform. It’s a significantly bipartisan issue, this isn’t impossible. The crux is do we get “transgender prisoners” Kamala or do we get “affordability” Zohran.
Please be aware the left has an anti- AI regulation faction as well.
Article: “Congressional Democrats in Hysterics About Saving the World From AI | Why panic when you could just use existing laws?”
I lean toward thinking these authors (Whitney Whimish and David Dayen) are closer to an ally, but it could go either way. If you argue with them, try to not unnecessarily crux on doom | not-doom, and try to not crux on federal-regulation | not. Seems like even if they’re wrong that federal reg is unneeded, they’re. right there are a lot of local and state power levers, so go ahead and explore that.
Love them or hate them, but, the left is very focused on power analyses of corporations. This is brainpower and organizing skill that would be a shame to not partner with.
Atlantic commentary article: https://www.theatlantic.com/ideas/2026/09/ai-regulation-anti-monopoly-challenge/688707/
Relatedly, The Atlantic did the argument, “Rogue AI won’t kill anyone but people might” https://www.theatlantic.com/technology/2026/09/kara-swisher-ai-destroy-humanity-atlantic-festival/688721/. This is curious to me—I’m like, maybe this is close enough now that we’re both doomy, but maybe not. Rogue-ness does cause some more acute panic, whereas evil CEO is BAU.
Double relatedly, I’m interested in doing some AI news aggregation, seeking a partner if noticing these types of movements is interesting to you. You know how Zvi does model card roundups, I want to do “public sentiment” roundups.
If x-risk is real, denying it means we die,
Only if the public and the labs simultaneously deny it.
We did an action in the direction of denial (Hawley, Cruz, Warren denied them their regulation), and the labs responded by saying they’re forced to slow down as they have not solved alignment (Sam on 9⁄16). That played out better for us.
Has anyone tried making an offsets slush fund donation that other EAs can draw from when these one-off situations come up? The intent of the donation would be to abet various causes EAs participate in but for which an offsettable hang-up emerges they can’t personally cover at the time. It would help keep in balance low-var and high-var.
Notes on food culture:
America swung back toward animal protein post-2024. Either correlated with the election or people just noticed around then. Some vegan restaurant and vegan food lines have gone under, it’s been a significant change. I’m not sure this is cruxy, just good culture knowledge generally.
reasonably high-end vegan Indian or Lebanese food at an American luncheon? You could maybe do this without weird looks. I wouldn’t because dietary ethic, but it’s in “ok maybe” territory for me now.
fast-casual like Chipotle, I’d avoid all-vegan in that case, yeah.
The specific claim: “don’t let China distill are models” is a regulation that would “help them at the expense of smaller competition”.
The general claim: corporations are ruthless optimizers toward the marginal dollar they can earn and will use any means at their disposal to achieve it. In particular they will capture or utilize government (i.e. lobby) to change the regulatory environment in their favor. OpenAI/Ant are extremely behaving as typical corporations. Their corporation-ness did not suspend because they are building unstable WMDs, in fact, $100 billions worth of investors saw them as a good place to invest their money while they were doing this.
“Negative leverage” is the business word, or you could stretch it a bit and say not dying is an inelastic good. If they have negative leverage over you, they will extract concessions. If the negative leverage is simple you dying, I’m going to call it a hostage situation. If they also believe in x-risk, then the game theory is a game of chicken, not utopic cooperation while we dig ourselves out of this. Are they criminals who are desperate for money? That cruxes on whether their hacking sprees were negligent or reckless.
Here’s my mental model: there’s an AIS faction and an investor faction. The investor faction is thrilled at every moment of this. The AIS faction thinks Dario is secretly on our side and Sam is maybe winnable. Here are things we do for the investor faction as free labor, hoping this will have some payout for AIS later:
vociferously defend their right to collude as long as they bring up safety first
launch a massive x-risk-awareness campaign for them trying to give them more negative leverage over the public
promote Dario’s regulatory framework
A lot of normies pick up on this immediately because they’re not x-risk-pilled, and they flatter us by thinking we couldn’t possibly be this… something, so they assume we’re simply on their side.
We have one victory: Sam was forced to believe in x-risk, which gave us leverage over him. This let Cruz/Hawley/Warren ignore his calls for regulation, and he backed down on 9⁄16 and admit he can’t just build the unstable WMD.
One of these two factions is getting played, although they will keep both of us in the dark on this until the very end. I think it’s us, because our strategy is something like, “p(doom) is so high that we should never call their bluff,” and that’s a bad strategy.
Demanding the U.S. government clamp down on Chinese IP theft, like every company has done since forever. One of their biggest priorities was perfectly predictable from anyone who has read an Economist article with an American company and China being on the same page. It was also independently predictable by the AIS community. Both models worked here but only the former is agreed on by your non-x-risk-pilled friend.
Yes, it means if they develop a new model it just leaks right away, so they are disincentivized to do that. If you’re not going to ban training you may as well not protect them for doing accelerationist stuff.
You know this is just a hostage negotiation where they’re using x-risk as leverage, right? Appeasing them makes the x-risk threat worse.
I would spend dialogue points aligning that x-risk is real. It’s a hard leap. There’s like 12 objections people go through. If they really crux on “but how will it leap to physical world?” or “I just don’t see why it would want to” or “it could cause a power outage, yes, but kill millions or billions of people?” then they’re not feeling it at the level that leads to setting aside differences.
If you do fully align, then at that point I assume I’m co-equal with them on policy, politics and I assume good intent if their ideas are weird to me.
Basically you’re deep canvassing, uh, horrifying someone. But the policy alignment really doesn’t matter much here. You want to recruit their brain cells and organizing efforts and relationships to the problem.
As for left-left: I’m going to stick by, align on x-risk above all else. Concede and find common ground on things like, yes it’s also (but not just) hype, they’re getting away with a lot, we’re underutilizing existing laws, rationalists a weird. But you haven’t really recruited their whole brain, mentality, movement, organizing structure if x-risk itself doesn’t click.