How is this different from what ChatGPT, OpenClaw are already doing? Claude is the one that pivoted more business-purposes than whole-user purposes.
less_raichu
Karma: 158
OpenAI’s board tried to shut it down around 2024, I think in reaction to the early reasoning models. Microsoft was then in the middle of acqui-hiring the whole company and the shutdown was canceled, and the board members trying to shut it down were ousted. So we tried this already, the result is the shutdown attempt just fails completely.
I originally suggested none of them just sincerely want to shut it down, but we pretty much have a stronger result already—we tried, it failed.
And given that your argument is good and no one is doing that, we can infer they’re just lying, and they personally want to plow ahead aggressively of their own volition. I just wish you stated this directly. We shouldn’t be in the business of saving the appearances for their commitment to x-risk reduction. And we really shouldn’t assume they’re anti- s-risk when they think they’ll be the godkings through this transformation.
My impression of top concerns, ranked: job loss, tech oligarchy is too powerful (vague/catchall-ish category but feels like its own vector), data center impact on environment both local/global. A few more: AI art specifically (replacing good artists with slop), dead internet, AI parasocialism, impact on education / kids, deepfake impact on democracy, inaccuracy/hallucinations rendering its main purpose useless.
So the crux may be that you think there’s not much anti-data center furor? Okay, so here’s my reasons for thinking people are extremely anti data center right now:
Reddit constantly (literally this is the third result when I opened reddit up to see: https://www.reddit.com/r/Wellthatsucks/comments/1u56euq/)
NY-12 primary, to the point that it’s a liability for Alex, since Alex’s opponent thinks Alex did not oppose data centers soon enough.
Politico has picked it up several times e.g. https://www.politico.com/news/2026/06/13/battleground-house-districts-data-centers-00952073, https://www.politico.com/news/2026/05/08/georgia-data-centers-water-00909988 - it’s also covered both D and R turning on AI
I disagree that for normies it’s about local impacts. That’s one of 7 concerns driving the anti-data center fervor, and they have rallied against data centers because that’s where they have the most power. Arguably their epistemic is kind of poor since it’s a big “reductio ad Hitlerum” where Hitlerum = AI, and yet, it coalesced around where they actually have power, so it’s not that reductive either. They really are just outperforming rationalists here and we should join them.
people don’t really organize around fear. it’s more like anger/justice on the negative end, and something like hope/change/feeling entitled on the positive end
the 20% versus 7% can translate to burnt credibility when coming from public health officials (this is part of the above)
fear can be spammy—you’re trying to induce a strong emotion in someone to get them to do something, which makes you not trustworthy.
This argument seems weak on two fronts.
There’s no reason to think LLM safety today will scale if the LLM undergoes RSI and/or scales to ASI. Or maybe the LLM will discover the ASI algorithm and code that and deploy it.
Today’s LLMs are misaligned, see Ryan Greenblatt’s recent post.
I don’t get why they’re negotiating a cheerful price in the end, as opposed to just a price. It sounds just like booking a consultation and it varies by industry/task whether that’s free or not.
It seems more analogous to getting a Ph.D. in altruismology. Some of you are researching insect welfare, some are doing malaria nets, most none are doing U.S. based donations.
I’m a man with amateur-athletic grip strength, like I dead lift and rock climb, and I can’t open many jars unassisted. Note the trope of a man opening a pickle jar for a woman often involves the man handing it to a second man, who opens it, then the first man says “I loosened it for you.”
Jelly jars, and those small jars of ginger paste require breaking the air seal by banging it with an implement. Probably there’s a better way, I learned banging.
I can think of some containers designed to require an implement—wine bottles, paint cans, beer bottles.
It seems important to point this out.
Probably stuff like Gatorade bottles fall in the annoying space of hard for women, easy for men, often opened away from home, and seems slightly unnecessary to be hard to open. But this space is definitely not as big as “jars”.
Can this methodology break the curse where disagreeable people are loudest on social media? Question-for-gift card people might have closer psychographics to Americans than social media posters and commenters.
For example—and AI use in art, and data centers, are becoming two rallying cries on Reddit—If I’m on Reddit and everyone is trying to boycott a video game for using AI, I might pop over to the survey platform and ask a question like “would you boycott a video game for using AI?” as a way to sanity check the Reddit opinion.
I don’t know how motivated rationalists are to work on “defending democracy” causes. I am becoming convinced authoritarians seem willing to do what unaligned AI wants, something I thought was kind of a joke or exaggeration, but when you have data centers competing for basic human rights like water and electric and winning sometimes, I’m less sure. This does suggest pro-democracy causes are pro-alignment.
I’ve thought of a snappy reply to the Skynet thing which is like—Skynet was sci-fi because we don’t give computer programs we don’t understand control of our nukes systems. It wasn’t sci-fi because there could be advanced programs we don’t understand. Well now we’re doing that.
It’s not meant to be deep it’s kind of pointing out, hey, you already agree whatever this tech is, we don’t completely understand it—that’s a much lower bar than “tries to break containment” and so forth—and we are doing many of the sorts of things you should not do with an ill-understood tech.
When I say “surprisingly open” I do not mean “I explain the paperclips thing then we align on p(doom)”. I mean things like: “we could lose control over it”, “the labs developing it don’t even know how it works,” “they’re going way too fast with no regard for safety,” “they’re in an arms race and I don’t know where it’s going.” Or more simply “I am alarmed, do you share in my alarm?” This is “surprising” because my 2025 prior is “it’s a word predictor” and “it’s a bubble.”
I also claim this is exactly where you want the conversation. Most of the heavy lifting is done. People will just demotivate if the conversation is about how everyone will die soon, and I don’t think “your vote has been estimated to reduce the chance of human extinction as much as 4 minutes of technical AI safety research” is going to work.
This would bite us if we had a credible “stop” movement, because a full stop will now cause recession. That would force a split between jobs-harm and extinction-harm. I will point out that political alliance building epistemics is mostly about UFO enthusiasts and astronomers figuring out they both want better telescopes, not aligning on p(aliens).
Explicitly using x-risk language, no, just me that I know of. Definite yes, to high appetite to talk about AI. Definite yes, to job loss, so that I think “mass job loss” is within public consciousness now.
There’s a fallacy/bias at play here. At some psychological or mythological level, people basically conflate “recessions” with “the apocalypse.” So “mass unemployment” and “lose control and kill everyone” are pretty near in the Overton window, so if we’re up to mass unemployment, we can start talking about “safety testing” even though that’s not directly relevant to unemployment.
Not mentioned in my post, and should have in retrospect, but Bernie Sanders has taken interest in AI-caused human extinction, explicitly, using numbers like “10% or 20% chance” in interviews. He is very respected with the left and with some smaller number of right-populists so it’s important to see if he’s successful in normalizing the topic.
Oh lastly—I came in with more canvass experience but numbers and conversation quality are pretty consistent between me, and the other AI safety motivated volunteers. The fundamentals are doing most of the work.
I’m surprised I can use phrases like “could lose control over” at all. Then I talk about a candidate who’s doing something closer to an AI safety agenda, including “accountability for mass damages”, instead of x-risk irrelevant priorities like banning chatbot medical advice.
I’m not pushing my luck… like I’m not asking people to agree to something like “I’m more worried about human extinction than unemployment”. But I’m discussing AI safety agenda, and getting there by saying I’m worried about “not controlling” AI. I think that’s un-mundane.
This is a political messaging problem more than a policy problem to me. “Stop AI” falls in the messaging category of “everything is bad but maybe we can make it get worse slower.” This is the same category as Biden’s “threat to democracy” and Kamala’s “he’s a fascist,” and it has a low ceiling in how much it can rally people.
Democrats especially need to be saying “we can have nice things.” Zohran figured it out with “affordability,” which Congressional Democrats have been trying to copy. The data center fight seems like a reversion to form: things are bad, but they should be bad in someone else’s backyard. Maybe the resistance politics is right here, but I don’t know why it’s so hard for Democrats to even suggest the possibility that the future might be OK.
It’s also bad policy to stop AI with no plan, given how much it’s already deployed, and how much capital we’ve invested.
So yes… maybe the center of the conversation simply should be “what should AI actually do,” then you stop doing the things that aren’t that.
(See Giridharadas “The Persuaders,” Ch 5, which is an extended interview with Anat Shenker-Osorio, who made this point about messaging that you need to sell a brighter future not a gloomy fight.)
It’s great it’s turned out enjoyable. I’m meeting up with some EAs later this week for street canvassing with some training and brainstorming—DM me for deets.
So far, I’ve been surprisingly effective/lucky in driving the conversations on the AI topic—maybe 90% of people want to go in-depth on that and follow me through. The other 10% of time it would have been nice if I figured out to pivot faster.
Part of the advantage of coordinating is we can pool together which issues come up and know what to talk about. “Buffer zones” is new to me—I’m guessing it’s pretty specific / sort of fringe, and you have some maneuvering room to get on your preferred topics.
ty! I guess from here I look if anyone’s connected open-world evals to gradual disempowerment, at least it’s more google-able now.
A minor point: just be really wary of putting down other volunteers, as “less agentic” or “more fuzzies motivated.”
I think it elevates us too much. It’s sort of a toxic trait to believe you don’t need fuzzies then other people just have to deal with making sure you get fuzzies. And just to say this part directly, it can turn into a sexist gender dynamic, when you remember D-side organizing volunteers are mostly women, and often we’ll be collaborating with people who have been doing this longer.
I’d emphasize “agentic group” is a different concept than “agentic person.” We’re all equally agentic to anyone else who showed up to volunteer but some of the multiplier effect is we’re also part of a forum community like LW where we read stuff like this, and we debrief with each other. It builds both skills and fuzzies.