Rattengift

Karma: 178

Rattengift 19 Jun 2026 7:57 UTC
2 points
0
in reply to: Tom Smith’s comment on: Gears for political races

Which of Lasher’s policies do you think are more progressive?

I seemed to remember hearing something about open support for rent control, which seems very popular in NY, but this could be misremembering on second thought!

My opinion was primarily informed by the polls that the NY Times has collected, including one funded by Bores that showed Schlossberg in the lead, and Wikipedia’ing all the candidates, as well as this May interview of Bores where he brags about paying for Anthropic and Google LLMs. Thank you for clarifying!

I have now lowered my estimation of Schlossberg’s chances of winning, while simultaneously thinking that that prediction market is probably incorrectly-calibrated (look at Conway’s positioning, when he has name recognition in polling).

I could be over-weighting the value of polling, though.

Rattengift 18 Jun 2026 20:05 UTC
−4 points
−17
on: Gears for political races
Reading the candidate’s Wikipedia page, it notes that prior to leaving Palantir (already a hard sell for progressives), he had apparently sexually harassed a coworker, and gotten formally reprimanded for it. This seems like one of progressives’ biggest “no-gos” these days, and muddies the timing beyond “I left because ICE,” which was his initial claim. Coupled with the funding from Anthropic, which would also seem to be a rather negative signal for most progressives, I’m curious:

What are the odds you’d give to your candidate winning over someone like Schlossberg, who’s had national name recognition for a while, and generally uncontroversial name recognition at that? Or Lasher, who’s already politically-established, seemingly advocating for more progressive policies, and polling pretty well?

I don’t have a stake in the outcome nor strong feelings about any of the candidates, I’m just curious on your perception of the odds, how Bores’s actions and press coverage have changed the odds in your view, and how the odds have updated with the canvassing you’ve been doing.

Rattengift 13 Jun 2026 20:54 UTC
1 point
0
in reply to: yams’s comment on: Rattengift’s Shortform
The thing about that comparison is that the K2000 is a ROMpler (and there were releases a few years later that made this workflow much easier), so the sounds you can get out of it depend pretty heavily on what you give to it, and how innovatively you crush those sounds.

Regardless of your views on the (excellent!) K2000, please work on bringing the AI thing to a speedy close in one way or another (convincing Eliezier to become an accelerationist and to write accelerationist blog posts is perhaps the most simple route available to you as someone who works at MIRI) so there can be another good Kurzweil synthesizer, please. This would meaningfully improve my life, and maybe lead to a new Postal Service record, and therefore has a higher EV than anything else MIRI could be doing right now.

Rattengift 12 Jun 2026 23:59 UTC
6 points
0
on: Rattengift’s Shortform
Everyone on this site knows Ray Kurzweil. Unfortunately, I think almost no one on this site knows the thing Kurzweil was responsible for that was most important: The K2000 synthesizer.

It had an incredibly impressive synthesis engine, and an interface so complicated that the “video training manual” was over an hour long!

The sounds that come out of the thing are absolutely transcendent, and if you were into music in the 90s or 2000s that wasn’t rock music, you probably have enjoyed it in something or another.

If you work on AI, please consider hurrying to one outcome or another so that Kurzweil can get back to doing what I care about (making beautiful and innovative ROMplers) prior to his eventual appointment with a liquid nitrogen vat. The new models simply are not the same, and the space could use his undivided attention once more.

Rattengift 11 Jun 2026 7:15 UTC
2 points
0
in reply to: Raemon’s comment on: Rattengift’s Shortform
I hope you write more often in the future. Annoyingly Principled People, and what befalls them has actually changed the way I’m approaching life in a pretty substantial way, by managing to convey a specific phenomenon in a way that felt very meaningful to me, and by causing someone in that thread to write a specific comment that did the same. Remains to be seen what the outcome will be to the changes I have made due to the post, but it has certainly changed things for me.

Something that would also be worth considering is a second-chance pool for the frontpage, consisting of posts from a long time ago. Occasionally resurfacing something from a long time ago to a lot of people at once is potentially very high-value; one could say that the feed is a variant of this, but the “lot of people at once” aspect is what I think is the most important piece of a second-chance pool. The feed doesn’t directly incentivize writing comments (since there’s significantly reduced visibility to old posts), but comments are a huge part of the value of a LessWrong post.

Rattengift 11 Jun 2026 3:08 UTC
1 point
0
in reply to: gwern’s comment on: “Programmer Science Fiction: My case for a new sub-genre”, Sam T. Oates 2026
I would probably offer something like ‘loopfic’ or ‘systems fiction’. Both because of the general ideological tie to Hofstadter’s work (as the essay notes), and because the most concise way of describing the pinpoint of the authors’ fixations is “the workings and immense failure cases of strange/recursive loops.”

It’s why, as the author notes, Infinite Jest and House of Leaves fit so well, despite not really fitting the S part of SF. The common ground between all entries that fit clearly in the genre is dealing with recursion (whether as an explicit focus of the story, like in AIfics, or implicitly, like in the sub-subgenre of libertarian governance fics) in a deeper way than most other genres. This approach to storytelling bleeds into nearly everything about the stories told in the genre, down to the interactions characters have.

Rattengift 10 Jun 2026 23:54 UTC
59 points
34
on: Rattengift’s Shortform
In 2011, Phil Goetz wrote the post “What we’re losing,” which bemoaned the shift from applied rationality to theoretical rationality on the site.

Regardless of the era we’re in, I believe LessWrong has gone too far in the other direction: To some loose approximation, every post is now about AI. Even assuming it’s what you care about more than anything, even coming from a prior of “the highest-EV move I can make is to write and interact with posts on x-risk,” you should heavily consider the benefits (both social and practical) of diversifying your intake and output.

Having norms around posting non-AI-related writing would be a good thing even for your ability to judge a person’s output around topics you care about more, like AI safety.

When the entire discussion is indexed around one topic (or a narrow range of topics), it becomes very easy to adjust the way you’re speaking and thinking to “fit” properly within the distribution of speakers around you. This allows for bad actors and poor thinkers to have an easier time hiding themselves within a group without revealing themselves as such. This is, obviously, bad: A poor thinker operating in good faith should have the opportunity to be corrected, and a community is far better off when it’s able to be mindful of which people are acting in bad faith.

This is sort of a plea: Please, write some silly fiction. Write posts about getting better at thinking. Write some posts to help people get better at taking more general action! There’s more juice to be squeezed out of the lemon of rationality, and when you’re feeling anxious and can’t stop staring at the end of the world, lemon juice will at least compliment the mood you’re in.

Rattengift 9 Jun 2026 0:02 UTC
2 points
1
in reply to: Oliver Kuperman’s comment on: Oliver Kuperman’s Shortform
It isn’t elitist nor privileged to say that writing skill matters. Raw intelligence isn’t a useful proxy metric for whether someone’s thoughts are valuable, but writing is, and this wouldn’t change if it were correlated with socioeconomic status. You just cited a chatbot’s output instead of backing up your own point. This demonstrates shallow thinking. The bot has diminished your ability to effectively argue for your position, and worse, it has gotten rid of something that would have helped you think better: Being forced to confront the existing data available yourself.

To quote a Richard Guindon comic strip:

Writing is nature’s way of letting you know how sloppy your thinking is.

Requiring people to have acquired the skill to write prior to using a text-based discussion site is the correct approach. There are virtually no benefits to allowing LLM-assisted writing and an endless series of cons.

Rattengift 8 Jun 2026 20:00 UTC
6 points
3
in reply to: Oliver Kuperman’s comment on: Oliver Kuperman’s Shortform
I think this is wrong. Helping people be less anxious in churning out words is bad. Perfectionism is neutral, but reducing people’s anxiety when writing is bad. “These words are written poorly” is an incredibly useful proxy for “this person doesn’t know what they’re talking about.”

Treating writing as a staple crop (like wheat) rather than a complex product (like chocolate milk) is pretty flawed: More isn’t always better. The steps taken to create it matter, and the work done to produce it materially changes the final outcome. You can absolutely ruin the final product by changing the process or substituting components (chocolate milk with artificial sweeteners tastes uncanny even to people who like artificial sweeteners; LLM text and text written predominantly by people who read LLM text feels bad to read, and is generally shallower).

Even if LLM text wasn’t worse aesthetically, which at the present time it seems to be, the end result of allowing people to use LLMs in writing would be worse text from authors who had less to say and lower standards. “Enjoyability of writing” is not something sane to optimize for, and LLMs actively harm most of the things that people value in reading. The fact that a human took a substantial portion of time out of their own life, to convey something that meant something to them to others, is a way to mitigate DDOS attempts on people’s attention, and the act of doing the labor of writing yourself implies you understand the topic in a way that isn’t demanded of you when writing with an LLM.

Rattengift 7 Jun 2026 5:39 UTC
26 points
9
in reply to: StanislavKrym’s comment on: leogao’s Shortform
Centralization is the natural conclusion, yes. The future that frontier labs are pushing toward is one of centralization of power, into the hands of people who have demonstrated little but carelessness. Anthropic’s early talent pool was seeded with Alameda Research (its largest investor circa-2023) employees; its Series B was led by a person who was deeply wrong about risk calculation and sloppy in execution with stolen money.

This is not existential risk, this is just risk. A risk that these companies (and governments, at their behest) are walking into wholeheartedly, knowingly, and happily. These companies are sacrificing the future using the language of utopianism while repeatedly advocating for centralization of power, which will only cement and calcify power structures.

If these people were acting in your best interest, they wouldn’t have been taking the same sloppy, anti-moral, deeply-flawed actions at Alameda, in an entirely different domain than safety research. They wouldn’t be repeatedly changing their RSP, removing all of its teeth. They wouldn’t be doing what they’re doing. They are telling you the world is ending to acquire casus belli, not to try and make it possible to stop the end of the world.

Even assuming ASI never happens, what exists and will come to exist in the next ten years will absolutely further cement existing power structures. It seems likely that the tools they have created up to this point are capable enough of dramatic increases in dystopian surveillance and government capability, and have been for a while.

I’m not entirely against x-risk argumentation; generally, I really don’t care either way (no stake in it; no influence on it). However, it absolutely is used to take frequently-unethical actions, and is self-justifying. “We, company, have to do thing, because we, company, are probably going to destroy the world with our current course of action. We’re aiming to, anyway.” Getting people to buy into their frame is part of their pitch deck. I think, primarily, they are careless people, who wanted to get justification to ignore law and ethics.

To quote an old George Rosenfeld post on the EA Forum:

When my friends in finance (not earning to give) tell me they’re working at Goldman to improve the world, I am normally sceptical. Psychology literature on motivated reasoning and confirmation bias suggests that we are excellent at finding views which justify whatever is in our interests. For example, one study shows that our moral judgements can be significantly altered by financial incentives; another shows that we naturally strengthen our existing views by holding confirming and disconfirming evidence to different standards.

Fortunately, unlike with finance careers, I think that longtermist careers are likely to be among the most impactful available to us. But given the financial incentives, I would expect it to be very difficult to notice if either longtermism as a whole or specific spending decisions turned out to be wrong. Research suggests that when a lot of money is on the line, our judgement becomes less clear. It really matters that the judgement of EAs is clear, so having a lot of money on the line should be cause for concern.

This is especially problematic given the nature of longtermism, simultaneously the best-funded area of EA and also the area with the most complex philosophy and weakest feedback loops for interventions.

Long-termism allows for sloppy thinking, financial incentives encourage sloppier thinking, and x-risk arguments tend toward that for this reason. There’s a reason so little is said about the concrete harms happening today by these companies in rationalist circles, and everything is said about x-risk: x-risk doesn’t demand you to change your behavior. It encourages you to reinforce it.

Rattengift 7 Jun 2026 3:47 UTC
13 points
−14
in reply to: leogao’s comment on: leogao’s Shortform
I think part of the reason for people leaning so heavily on x-risk arguments is that the alternative (that they created machines that are somewhat uniquely destructive to labor, are incredible for surveillance, destroyed the free software movement, and ruin something that absolutely all of them love, the internet, out of a mistaken belief that if they didn’t, a paperclip maximization machine would be conjured into existence by someone else, and it’s beneficial to be the first one to make the paperclip maximization machine) requires them to stare in the face the rather dreadful implications of their actions, and figure out a way to salvage them.

When operating in an emergency mode, you sort of get to ignore ethics of anything immediate in favor of the hypothetical ethics of something that may or may not come. That’s why Anthropic allowed itself to be integrated into Project Maven (responsible for thousands of deaths until it became a PR issue), why they automatically offer all of their models to the NSA, why they constantly push for unethical sanctions on China, why they’re paying billions to Musk, and why they turned Bun into a security nightmare: Because you gotta go fast, or you’ll lose. And losing is bad. For x-risk reasons. They’ve created a maximizer for GPUs and market valuation instead of paperclips.

X-risk is poison for consequentialists; a complete hypothetical encouraging them to create the ultimate puppy-kicking machine.

Rattengift 7 Jun 2026 3:24 UTC
3 points
0
in reply to: leogao’s comment on: leogao’s Shortform
One possible approach would be having someone else polish the posts for you/work iteratively on ideas with a friend. Writing partners are useful! Not only do they divorce you from some of the reputational risk of writing whatever you’re writing, they also turn the worst part of writing into a non-issue (the mental context switch between generating and polishing). There’s also the concept of beta readers, but that admittedly feels a bit sillier for web forum posts. Just being able to offload the mental work of perfection into a “good enough” of “$person i trust thought it was good actually so it’s fine” does a lot for improving speed of iteration, usually without meaningfully degrading quality in a noticeable fashion.

Rattengift 5 Jun 2026 3:31 UTC
1 point
−4
on: Rattengift’s Shortform
I have been revisiting the 2023 post “AI Timelines” today. I would be interested in seeing what the participants within would offer as estimates at the present time.

From my own cursory estimation, it seems like Kokotajlo was wildly miscalibrated on many or most fronts, while Cotra and Erdil’s predictions in 2023 seem fairly calibrated, leaning heavily in favor of Erdil (who seems in retrospect to be remarkably well-calibrated).

Scattershot takeaways:

Kokotajlo correctly assumed governments would completely fail to slow down timelines.

Erdil’s predictions are interesting; he speaks like a gambler (in a very positive way); very precise, taking into account many systemic factors, while not allowing any one to dominate. Conservative revenue estimate in median prediction scenario for labs in 2030 (which seems reasonable given how historically unprecedented the scale of the current economic bubble has been), with seemingly-reasonable predictions across the board otherwise.

Cotra quote that I thought was fairly solid:

Yeah, I just think the way we get our OAI-engineer-replacing-thingie is going to be radically different cognitively than human OAI-engineers, in that it will have coding instincts honed through ancestral memory the way grizzly bears have salmon-catching instincts baked into them through their ancestral memory. For example, if you give it a body, I don’t think it’d learn super quickly to catch antelope in the savannah, the way a baby human caveperson could learn to code if you transported them to today.

I think the ultimate conclusion I have is that habryka needs to do more interviews. He’s good at them.

Rattengift 1 Jun 2026 7:49 UTC
1 point
0
in reply to: G Wood’s comment on: G Wood’s Shortform
It’s like spitting in someone’s food: They may not notice; the individual instance might not cross a meaningful threshold of harm. It’s still wasting their time, risking harming them (you didn’t care enough about the text to write it yourself, which implies a lot of qualities about it) and forcing them to do something they’d likely rather not, were they properly informed of what was going on. It’s just rude, and significantly more likely to be harmful to the reader than human-authored text.

Rattengift 31 May 2026 20:42 UTC
0 points
−1
on: Pursuing the target
This was really cute! Good job!

Rattengift’s Shortform

Rattengift28 May 2026 10:20 UTC

2 points

10 comments1 min readLW link

Rattengift 28 May 2026 10:20 UTC
1 point
−2
on: Rattengift’s Shortform
Today in concerning news about people who should know better:

“One night I quietly gave our AI agent full access to YC’s production database. It made the agent 10x more useful. That’s what convinced me that trust-by-default is the only way to get the most out of agents.”

The current socioeconomic moment is just so.… stupid. Meritocracy and just-world fallacy have always been fake, but what exists in their place has rarely been rendered as transparently as in modern times. The bright side is that these people clearly have too much money, and they are not so bright that you will find it hard to relieve them of that burden, if you set your mind to it. The downside is that these people are not competent in their evil, which is, strangely and counterintuitively, so much worse than the alternative.

Rattengift 26 May 2026 13:34 UTC
1 point
0
in reply to: Jay Bailey’s comment on: So you don’t want everybody to die
Thanks for the acknowledgement that I’m not hitting a brick wall! I’d happily die on any hill, ^[1] but it’s always more enjoyable when dying on it to open ears.

The reason I was exposed to arguments around x-risk is the same reason that I’d imagine most people who have been actively making the “problem” worse know about it: It’s something that a fanfiction author who was unusually effective at popularizing ideas in nerd communities while being technically incompetent has been obsessed with for nearly thirty years. The reason I forced myself to think on it enough to feel okay arguing against it was due to being successfully convinced about EA’s value when I was a child, ^[2] keeping up with it in adolescence, and noticing that x-risk-mitigation advocates successfully started diverting a bunch of funding from malaria nets to “extinction prevention,” by making arguments around EV, which in my mind (after much consideration) is likely going to go down as one of the biggest unforced mistakes in human history. This is neither here nor there, though; ^[3] just relevant background.

This arms race framing is, transparently, used by major labs as a way to justify and provoke their intense valuations and immense spending on attempting to build the thing that you’re worried about. It’s a way of red-teaming venture and human capital. “The first person to build it controls the entire future, so we need to have a good actor doing it. Give us ten trillion dollars.” ^[4] The only person to have participated in the founding of nearly every major lab (Musk—OpenAI->Anthropic, xAI) publicly acknowledged the fact that he met his ex-partner off a discussion of Roko’s Basilisk. More people are familiar with the argument in its primary form of “justifying bad actions from bad actors” than its secondary form.

I’m not making an argument that you should stop talking about extinction vis-à-vis lawmakers. I don’t think it’s as persuasive as you’re making the case that it is, and I think that there are many things lawmakers largely agree on but don’t have the political will from the wider public to execute on, but I don’t fault trying to convince lawmakers as a strategy in itself. However, the graph you linked earlier just shows awareness of the argument, it doesn’t show alignment or agreement with it, nor whether any of the lawmakers who have made those arguments would actually pass policy proposed to mitigate x-risk. It also (I believe) is likely going to look a lot worse after midterms come around.

There are five hundred and thirty-five members of Congress; thirty are talking about, not necessarily aligned with, the concept of AGI risk. Even assuming the current trend scales linearly, which doesn’t seem likely to me, it still doesn’t seem to be enough with the timelines the people who made the graph are projecting. Appealing to populism to pass stalling legislation, given how far ahead negative views of AI among the public are relative to lawmakers’ views on artificial intelligence, even if your goal is still ultimately a complete ban, would if nothing else likely buy you the time to convince lawmakers of what you aim to convince them of. Buying yourself another 5.5 months, under the implication of the graph, would literally double congressional awareness, being dramatically more effective (x2 of the end result of your efforts without it) than just trying to convince lawmakers, and reducing the effectiveness of LLM companies using populist policy and regulation would likely offer significantly more time than an extra 5.5 months. Popping the bubble would take a lot of people’s compensation with it, pushing the economy to reallocate resources to more productive uses of time than literally killing everyone.

My claim is more that efforts to convince the wider public, not lawmakers, has been mislaid, and that the energy would be better shifted in a marginally different direction. You’re unlikely to successfully convince the wider public; they’ve been continuously inoculated against what you want them to believe, by the ever-persistent, ever-aware-of-x-risk voices of the people currently causing the problems at hand. If the people making an argument are people you absolutely despise, who you can see speak from both sides of their mouth, ^[5] it’s far less likely to gain any ground in your mind. This has been the public’s current exposure to x-risk arguments: They exist to increase animosity to generative AI, due to the sheer hypocrisy of the players involved making them, not to gain purchase due to merit but rather blunt force of capital.

It’s also worth acknowledging that this post was, admittedly, a very silly way of talking about the art of convincing people in public in general, quite literally inspired by a street I walk on every day being filled with literal petitioners, who are trying to achieve much easier things, who still often fail at what they’re trying to do, because they don’t understand how persuasion works. It applies to lawmakers, as well, but lawmakers are closer to you than you are to the public. The “Save BART” people are actually a good example of inefficiently trying to compel the public into action (though I wouldn’t be surprised if they succeed anyway); it is nearly impossible to convince people to sign a petition off of love for something, but it is extremely easy to convince someone to do something out of spite. Fear is somewhere in the middle; accepting that you’re afraid is in a sense admitting to cowardice. Most people are averse to this, unless it’s really bad, so it’s very hard to convert fear into action. Easier than to convert love into action, though.

The strategies to use in private are different than the ones to use in public, so the post doesn’t bother to target that use-case. I would likely write a different post if writing advice specifically for your scenario of convincing lawmakers, but I’ve never been around lawmakers to verify my suspicions. It does sound like a fun game, though; at some point I should figure out something fun to try and lobby for, do so, and then write that post after I’ve proven my point. It seems likely that you’re not doing something correctly, if there are only thirty American Congresscritters who have bought in (and seeing how success leans much heavier toward the American right-wing in recent memory, per the graph), but I can’t really diagnose it without either seeing what people are trying or trying to do it myself and seeing what the failure cases are. I have some existing priors about how Congressional persuasion happens, based on what I’ve read from and of people who have successfully done it (not in the x-risk space), but I have no way of validating my suspicions around it at this time.

The post never actually encourages lying, it merely encourages not leading with the high-friction parts. Convincing people, especially in public, is primarily about reducing the friction to agreement. ^[6] Early existential risk arguments caught on largely by this method, by making it high status to signal agreement with it. In the awkward middle period we’re in, making it profitable for high-value targets is the current meta. However, status games don’t work as well on the public at large, and there’s not enough money to go around to incentivize everyone in the way the movement is currently seducing high-value targets with.

Having moved to SF recently, I’ve been made pretty viscerally aware of how rationalism has sort of failed in a place that should hold its apex. Everyone speaks with a rationalist tongue, but most of them only stole the rhetoric from it. People who should be halfway to your beliefs are instead of the belief that AI will somehow, or is, making everything better; describing silly fantasy scenarios of benevolent futures while consciously rejecting your view and mistakenly thinking mine is novel. It’s pretty puzzling to observe firsthand.

I think it’s primarily due to the public acknowledging that there’s something compelling about rationalism based off of interaction with rationalists, but failing to actually grab onto any of the thinking associated with it, largely due to poor public communication on behalf of the rationalists rather than active rejection. This, plus motivated reasoning with an assumption that the efficient market hypothesis is correct, has made their thinking even sloppier than it would have been without the exposure to rationalism. This seems bad to me. To convince people of a point, you don’t need an in-person conversation. In-person conversations make it a lot easier, but people successfully convince the public of things all the time. The failure of public communication on this axis feels a bit like a critical mistake, and one that was probably avoidable with different messaging.

Convincing guests on the JRE was fine, but the fact that they’ve been talking about it for years now on what is one of the biggest soap boxes in the world and have convinced exceedingly few regular people is a sign that the public messaging just isn’t working, and manufacturing public support is important to passing legislation.

Rationalists should win, and while I’m not a rationalist and I don’t care too much about winning, ^[1:1] I do think the strategy outlined in the post offers a much better approach to public messaging for rats, if they actually want to win in the long run on this issue, than the present approach, which is actively self-defeating.
1. ↩︎↩︎
  To quote a piece of art I like, “I think I’m right; I don’t think it matters.”
2. ↩︎
  circa nine years old around 2013 or so, which I’ll admit is probably out of distribution for exposure to x-risk arguments; I am not necessarily representative of the normal mode of discovering x-risk arguments. I consider myself to be fairly normal, though, as an individual.
3. ↩︎
  I will never forgive x-risk mitigation advocates for reducing my ability to use em dashes in public writings by virtue of causing the companies responsible for the creation of LLMs to get funding. I miss using em dashes. This is objectively worse than human extinction would be. I wanted to use one here, but, well, you know. I guess I could start misusing spaced en dashes, but it isn’t the same.
4. ↩︎
  Does this seem familiar? Which lab head am I referencing? Just kidding; it’s all of them, from Altman to Legg ^[7] to Amodei to Musk. They all make exactly this argument in the same hushed and worried tone while excitedly ushering in the same future they’re allegedly afraid of.
5. ↩︎
  A quote that comes to mind with how these people have historically advocated for their position is Machiavelli paraphrasing the Romans: “As for that which has been said, that it is better and more advantageous for your state not to interfere in our war, nothing can be more erroneous; because by not interfering you will be left, without favour or consideration, the guerdon of the conqueror.”
6. ↩︎
  In private, or semi-private contexts, strangely, it’s often the opposite. It’s very useful to start with something outlandish, the least-charitable-to-yourself way of presenting your views while still remaining honest (to quote Yudkowsky paraphrasing LeGuin, “Don’t say things that are literally false.”), that you’ll never in a million years convince someone of, and make a game out of convincing your target the halfway point is reasonable that way. This doesn’t really work in public, but I think it’s why Yudkowsky leans so heavily on it in public: It works really well in a semi-private context like 2014 lesswrong, or in the comments of a −14 2026 lesswrong post. ^[8] There’s probably a decent post to be written solely about this weird quirk of persuasion, maybe titled something like “Boiling the Frog” or something of that nature, but I think mechanically, if not getting into the “why” of how it works, it’s simple enough to be described by taking a relatively popular internet macro quote describing a bad faith action, and just generalizing it a bit beyond bad faith.
  
  I am, of course, referring to the Moxon quote that may have came to mind:
  
  Meet me in the middle, says the unjust man.
  
  You take a step towards him, he takes a step back.
  
  Meet me in the middle, says the unjust man.
  
  This doesn’t have to be an intentionally bad-faith action, and doesn’t actually require the act of stepping back to still be successful. Zeno’s dichotomy paradox points out that there are an infinite number of half-steps to get to any point. By repeatedly arguing your point, but staying stable in it, you draw your target ever-closer, even as you stay still and continuously take good faith effort.
  
  I think there are two types of natural reactions upon first exposure to lesswrong/Bayesian rationalism among people who eventually find themselves convinced of what’s within; there are people who find the tenets of it obvious, and there are people who find themselves with an initial aversion to it in a way that reflects the sort of mindset you’d have in a debate club, that dissolves upon further exposure and recursive argumentation. This latter mechanism, I think, though foreign to me (when I was exposed to rationalism I was very much in the former camp), is ultimately the same one as what I’m describing.
7. ↩︎
  Who, coincidentally, has been projecting 2028 as his 50% confidence target for AGI and has been on the existential risk train since at least 2011.
8. ↩︎
  It also happens to be really fun, which is motivating in itself, beyond anything about efficacy, and perhaps in conflict with it: I know that I, for example, will often argue in public in a way that’s primarily useful in private, just because it’s a lot more fun, to the detriment of the point I’m making.

Rattengift 26 May 2026 3:27 UTC
5 points
4
in reply to: Jay Bailey’s comment on: So you don’t want everybody to die
Sorry for the delay in response; I wanted to figure out how to word this correctly, so I let it roll over in my mind for a few days.

Sanders is usually on the right side of history. He was never not going to understand a rational argument. The end-goal of persuasion can’t just be getting people who were always going to agree with you on board in a democratic system. It has to be getting people who won’t to align with your goals despite their disagreements.

Human extinction isn’t that far out of the Overton Window at this point; people like Musk creating big headlines talking about it every year has moved it firmly adjacent to it, if not inside. It’s half of what gets talked about nowadays on some of the most-listened-to platforms in the world, like the Joe Rogan Experience, whose audience is most certainly not an educated one. It’s disagreed with, in part because it’s constantly used by people who are making the problem worse as a justification for their own actions and casus belli for worse ones (Amodei pushing for sanctions on China while becoming infinitely wealthy partially on the reputation he gained from working at a Chinese company, Musk’s blatant self-dealing, Thiel hesitating when asked if the human race should survive on camera; admittedly this last one is a bit unfair to Thiel, since we all know what he meant, but it’s nonetheless how it was perceived by a large audience).

I think you’re seeing it in the wrong way; the goal is not to “make people negative towards AI,” (which is happening and is going to increasingly happen regardless of efforts), it’s to channel the already-preexisting and increasing loathing ^[1] for it to achieve something productive. It’s simple coalition-building, which is how any functioning democracy has worked nearly forever. Making these companies less efficient and worse by heavily regulating them would buy you more time, and give the wider public what they want to begin with.

It’s something you can learn from the history of all of the recent ratfucked pieces of progressive legislation: Channeling good intentions and productive energy into something nebulous is sort of the vibe of the last twenty years. Does anyone like paper straws? No, but someone took existing frustrations over something (a lack of teeth to environmental regulations), and funneled it into concrete policy. The wrong policy, but policy nonetheless.

Energy and anger can be directed into things that sound plausible. A human extinction argument made by people who are enthusiastic about the technology doesn’t feel plausible; a “your life is going to get made worse by these companies; we should stop their ability to {monetize access to models, sell derivative works of IP that they don’t own the rights to, monopolize access to HPC resources, create the most useful technology for surveillance dragnets in the history of humanity}” pitch is plausible, and something a lot of people are naturally sympathetic to.

To paraphrase an old post by lc, “Thirdly, even if it’s going to happen tomorrow, every day that [AGI doesn’t exist] under this paradigm is a gift. Delaying a hypothetical [extinction event] means increasing the life expectancy of every living [being].” Pushing back timelines has an extremely high expected value, if you actually believe in the AGI scenario; limiting the ability of these organizations to move forward efficiently by any means necessary could make the difference.

Of course, I don’t believe in the existential risk scenarios put forth. I don’t think most people who claim to do so happen to do so, either, given how little anybody is actually seemingly doing about it. The urgency that a plausible x-risk scenario would imply just isn’t reflected in the actions or rhetoric of anyone currently speaking about it. Nonetheless, I think they deserve as much as anyone else the chance to achieve their goals, hence the post.
1. ↩︎
  NBC surveyed a group of registered voters and found that the concept of artificial intelligence was so unpopular that 46% of respondents said they hold negative feelings towards the concept, and only 26% reported positive connotations, while 27% were neutral. https://gizmodo.com/people-hate-ai-even-more-than-they-hate-ice-poll-finds-2000731438

Rattengift 26 May 2026 0:38 UTC
1 point
−5
on: Pope Leo’s First AI Encyclical – Summary and Commentary
This shortform post from Linch on it plausibly being partially LLM-generated is probably worth mentioning; it seems interesting and relevant to the subject matter at hand for just about anyone who would want to invest time into reading the encyclical.

Rattengift

Rat­tengift’s Shortform

Rattengift’s Shortform