Ebenezer Dukakis

Karma: 835

Ebenezer Dukakis 3 Jun 2026 16:11 UTC
1 point
0
on: Trees are mostly made of air and a generalizable lesson for AI safety

dry wood is mostly CO2

FWIW, you appear to be confusing two claims:

“Most mass in trees was previously part of the surrounding air”

“dry wood is mostly CO2”

From a US Forest Service website:

Wood is best defined as a three-dimensional biopolymer composite composed of an interconnected network of cellulose, hemicelluloses and lignin with minor amounts of extractives, and inorganics. The major chemical component of a living tree is water, but on a dry weight basis, all wood cell walls consist mainly of sugar-based polymers (carbohydrates, 65-75%) that are combined with lignin (18-35%).

https://research.fs.usda.gov/treesearch/42245

“Trees are mostly made of air” (direct quote from post title) is more clickbait headline than scientific fact.

Ebenezer Dukakis 10 Mar 2026 8:37 UTC
−1 points
0
in reply to: RobertM’s comment on: Raemon’s Shortform Feed

Yes.

OK so from my perspective, this favors my point then? You seem to agree that guiding Claude Code to produce a top quality social science paper is fairly possible. You haven’t given any particular reason to believe that social science work is fundamentally different-in-kind from AI safety work—indeed I expect there is a fair amount of social science which could be relevant to AI safety! We both agree that many naive LLM posts are crap. So why would I invest weeks or months in prompting and guiding Claude Code for alignment research, if my post will get placed in the same bin as the the “naive LLM crap”? Can I pay some sort of karma fee to be placed in a different bin? Can users with at least 100 karma have some other sort of “trustworthy LLM use” credit account which gets “overdrafted” if I am found to repeatedly produce LLM crap?

I do understand your concern.

Thanks for saying this. Sorry if I’m repeating myself too much.

if we enter a regime where LLMs are able to get useful[2] alignment research done in a basically automated way, then frankly I think we will have entered a completely different regime where we will need to be rethinking quite a lot of how we relate to the world.

A lot of credible people are claiming that diffusion will be a rate-limiting step on LLM adoption. “The future is already here – it’s just not very evenly distributed.” In my view you are thinking too much in terms of binary “regimes” and too little in terms of seizing opportunities when they arise.

80% it comes from labs first, so frankly I am not that worried about the second hoop.

OK but the stuff I’m seeing online about automated production of academic papers is coming from academics, not labs. The value of automated alignment research seems high enough that we should encourage random academics to contribute to it, if they believe they have a contribution to make?

Once more I implore you to look at the list of rejected posts on our moderation page and tell me that you think the signal to noise ratio would be improved by allowing unmarked LLM content on LessWrong.

Are we talking about this page ? Based on a quick ctrl-f for “LLM Writing”, the current policy has been invoked manually around 12 times in the past 6 months, with the vast majority of invocations being automated. It looks like there were 507 accepted posts in February alone based on https://www.lesswrong.com/allPosts ? So currently, under 1% of posts are manual LLM-rejections? From my POV, up to 10% of LLM posts would be plausibly be worth it for VoI purposes.

The automated LLM detection is likely a valuable signal, but it’s quite compatible with my advocated policy of filtering based on content rather than filtering based on the honor system (as are manual rejections!)

N hfre fbcuvfgvpngrq rabhtu gb cebqhpr n gbc dhnyvgl cncre jvgu na YYZ pna yvxryl nyfb qrsrng YYZ qrgrpgvba. uggcf://kxpq.pbz/125/ Qba’g rapbhentr crbcyr gb qrsrng YYZ qrgrpgvba, whfg znxr vg fb gung gur yrggre bs gur cbyvpl fnlf vg vf YYZ qrgrpgvba juvpu znggref (abg YYZ hfntr cre fr), fb vs na ubarfg crefba svaqf n jnl gb nhgbzngr uhzna-vaqvfgvathvfunoyr nyvtazrag erfrnepu, gurl pna or erjneqrq jvgu n ybg bs xnezn sbe qbvat fb. (Va bgure jbeqf: qba’g arprffnevyl nfx hfref gb hfr gur ubabe flfgrz va ynoryvat YYZ-bevtvangrq pbagrag; fvzcyl hfr lbhe rkvfgvat nhgbzngrq qrgrpgbe gb ynory cbfgf sbe ernqref, cyhf ghar vg bire gvzr gb cevbevgvmr “penc qrgrpgvba” bire “YYZ qrgrpgvba” cre fr.) (V’z znatyvat guvf cnentencu jvgu ebg13 gb znxr vg uneqre sbe qhzore YYZf gb svaq, gb uryc “novyvgl gb qrsrng YYZ qrgrpgvba” fgnl hfrshy nf n dhnyvgl fvtany.)

Anyways my position is not simply “allow unmarked LLM content on LW and let it rip”, I’ve already elaborated a number of alternatives in this thread. I don’t want to become a broken clock so I’ll just encourage you once more to brainstorm and evaluate alternative approaches here. It seems you will have to solve this problem “for real” eventually regardless of what you do. If you’re going to deploy the planned change, I encourage you to see it as a stopgap and start thinking about what’s next right away. Best of luck.

Ebenezer Dukakis 10 Mar 2026 6:03 UTC
1 point
0
in reply to: RobertM’s comment on: Raemon’s Shortform Feed

Doesn’t seem at all ruled out by what I said (though even then I do not think you could reliably do that without an expert human in the loop).

If I do this with a human in the loop, it will still count as LLM-generated and you will require it to be tagged as such, correct?

I am confused by what you think we’re currently doing and what we’ll be doing in the future.

I think you are currently prohibiting LLM writing and you will soon require it to be tagged as such, which will still de facto stigmatize experimenting with automated alignment work and nudge the leading edge of “LLMs-for-research” elsewhere. You’re forcing people to jump through two hoops: (a) produce good automated alignment research, (b) convince people it’s worth a read even though it’s AI. I’m saying (a) should be enough. The skills to accomplish (a) and (b) may be very different btw. And the best people at (a) are not necessarily the people who are already ingroup such as Ryan Greenblatt.

Ebenezer Dukakis 7 Mar 2026 9:18 UTC
1 point
0
in reply to: RobertM’s comment on: Raemon’s Shortform Feed

most of the value is not coming from the 200-line instruction file, it’s coming from the human judgment doing selection/pruning/etc.

It’s still early days at this point. I don’t think anyone can be confident regarding the potential quality of selection/pruning/etc. possible given the current technology.

I think if I was going solely based on your comments, I would not believe that creation of a paper that could go in a 1st quartile social science journal is possible with current technology. Yet experts appear to believe it is, in fact, possible. I would encourage you to practice the virtue of lightness on this topic:

The third virtue is lightness. Let the winds of evidence blow you about as though you are a leaf, with no direction of your own. Beware lest you fight a rearguard retreat against the evidence, grudgingly conceding each foot of ground only when forced, feeling cheated. Surrender to the truth as quickly as you can. Do this the instant you realize what you are resisting, the instant you can see from which quarter the winds of evidence are blowing against you. Be faithless to your cause and betray it to a stronger enemy. If you regard evidence as a constraint and seek to free yourself, you sell yourself into the chains of your whims. For you cannot make a true map of a city by sitting in your bedroom with your eyes shut and drawing lines upon paper according to impulse. You must walk through the city and draw lines on paper that correspond to what you see. If, seeing the city unclearly, you think that you can shift a line just a little to the right, just a little to the left, according to your caprice, this is just the same mistake.

Regarding investment of effort:

And current LLMs are not actually good enough to act as a reliable filter in a way that can be scalably automated, at least not with amounts of effort that make sense to invest given the steady march of progress.

I think it could make a lot of sense for someone to invest that effort, if they want to create differential technological development in favor of alignment research. I expect your current course of action will effectively take that option off of the table in practice.

If you wait until it’s manifestly obvious that AI agents can help with research, you’re essentially creating differential technological development against AI alignment research, relative to other field that continue to publish solely on the basis of quality.

If you must discourage AI, at least you could use some sort of very sensitive leading indicator such as a weekly competition for best AI-generated alignment post, so AI posts are encouraged the instant the technology is good enough to help us with alignment. (Specifically, each week the winner of the previous week’s competition could be submitted masquerading as an ordinary LW post to see how it scores, so that way you’re collecting 1 data point per week about the state of the technology, while allowing a max of 1 AI “spam” post per week.)

But instead I would suggest you simply discourage posts if you can tell it’s AI, rather than using the honor system.

Ebenezer Dukakis 6 Mar 2026 6:52 UTC
1 point
0
in reply to: RobertM’s comment on: Raemon’s Shortform Feed
Why did you believe that the posts you’re currently rejecting would affect my take? What are the points of contention in your view?

How many of the posts you’re rejecting do you believe were created as described in e.g. this post?

When I put more effort into prompting and did a “day-long” back-and-forth with Codex 5.3 (extra high), where I have a whole pipeline with R, LaTeX, custom skills files, and so on, the final output was of course much better; something that could probably land in an average 1st quartile social-science journal. From my perspective as an academic researcher, that’s wild. (I put day-long in quotes because there was a lot of free time in between my prompts and the agent working. I just had to check in once in a while.)

https://statsandsociety.substack.com/p/you-should-absolutely-be-freaking

From a quick glance, the posts you’re complaining about looked like reformatted ChatGPT conversations which use less than 0.1% of the compute cycles of the hypothetical posts I’m arguing could have good insights.

It doesn’t make sense to have a single generic AI tag if some AI posts use 10,000x as much computation as others.

Ebenezer Dukakis 6 Mar 2026 0:07 UTC
−1 points
−1
in reply to: habryka’s comment on: Raemon’s Shortform Feed
Different people will use different AIs in different ways. You’re potentially removing the incentive to figure out how to do good AI-assisted alignment work, since it’s known in advance that there is no payoff since no one will read the post simply because it is AI.

Ebenezer Dukakis 6 Mar 2026 0:01 UTC
1 point
0
in reply to: RHollerith’s comment on: TsviBT’s Shortform
So what I’m hearing is that we need to ban AGI, plus ban any geopolitical play which could create an incentive to violate the ban and create AGI, plus ban any geopolitical play which could create an incentive for any of those geopolitical plays, plus ban any geopolitical play which could create an incentive for any of those geopolitical plays...

Ebenezer Dukakis 5 Mar 2026 20:55 UTC
1 point
0
in reply to: RobertM’s comment on: Raemon’s Shortform Feed
You’re confident that you couldn’t build an AI agent to flag posts with quality issues? I can help you brainstorm in DMs if you want.

Ebenezer Dukakis 5 Mar 2026 19:52 UTC
1 point
−1
in reply to: Raemon’s comment on: Raemon’s Shortform Feed
A high-quality alignment post is novel, important, anticipates and answers possible objections, doesn’t have major reasoning flaws, etc. etc. It feels very possible to construct AI agents to check for each of those. Helps sift through human-written posts too.

we’re certainly at least going to be spending one generation of frontier-model where it’s only doing the latter.

I think it’s a mistake to view generations in this discrete way. Academics on Substack are talking about how today’s tools seem sufficient to automate paper production:

But effective use requires investment: Aziz Sunderji describes building a ~200-line instruction file encoding his research workflow, judgment calls, and behavioral guardrails. This takes a skill.

You seem to be saying that if someone like Aziz adapted their 200-line instruction file for alignment research tomorrow, you’re not really interested. I would be doing the exact opposite, I would be begging him to adapt it.

If the LW team chooses to be on the lagging edge of this wave, as opposed to the leading edge, it could be the last mistake you ever make.

Think of it in EV terms.
- Suppose there’s a 90% chance you’re right, and a 10% chance I’m right.
- Suppose if I’m right, and you follow my advice, we get +1000% alignment insight production. From the linked article: “David Yanagizawa-Drott has taken things further still, launching a project to produce 1,000 economics papers with AI—not as a stunt, but as a stress test of what happens when the cost of generating research drops to near zero.”
- If I’m wrong, and you follow my advice, we get 10% drag due to poor filtering of AI slop. (Bad posts are still downvoted, good posts are still upvoted, so overall it’s just a bit of drag.)
- So the EV is 0.9 * 0.9 + 0.1 * 10 = 1.81, +81% research speed on expectation.
I tried to be quite generous for you with those numbers, but maybe you disagree.

Note also that if the experiment with allowing AI is going poorly, you can always roll back/change course/adapt. But since LW is essentially the main venue for alignment research, by taking a hardline stance against AI, I see you as basically deciding that alignment research will be one of the last areas to be revolutionized by AI paper production. I just don’t understand why you would make that choice!

I think prohibiting AI posts only makes sense if you believe we are basically on track to solve alignment and the important thing is to avoid rocking the boat. On the other hand, if you think x-risk is high, maybe it’s time to play to your outs.

A compromise approach would be to have a weekly or monthly competition for “best AI-generated alignment post, as judged by AI” (and restrict AI usage outside that competition). So you’re still tilting the field against AI posts but at least you’re no longer constraining yourself to the lagging edge as badly.

Ebenezer Dukakis 5 Mar 2026 16:20 UTC
4 points
0
in reply to: TsviBT’s comment on: TsviBT’s Shortform
The ban on space nukes doesn’t seem to be looking good

NATO is concerned that Russia is seeking to deploy nuclear weapons in space, a development that could threaten the thousands of satellites orbiting the Earth that are crucial to defense as well as people’s daily lives, the alliance’s chief said.

https://www.politico.eu/article/nato-chief-is-worried-about-russian-space-nukes/

Ebenezer Dukakis 5 Mar 2026 15:45 UTC
4 points
0
in reply to: Raemon’s comment on: Raemon’s Shortform Feed
I often paste a comment draft into an LLM chat and ask it to flag issues with spelling, grammar, or phrasing. (I did it with the grandparent of this comment for example, and accepted a suggestion or two.) If I choose to adopt the LLM’s suggested corrections or rephrasings, do I now have to delimit them in the AI tag?

How about use of AI for anonymization purposes such as the recent Possessed Machines essay?

Forcing people to disclose grammarly usage feels like petty authoritarianism. I still think you should focus less on usage of AI and more on the actual content. Does it feel like AI slop? Is it not “testimony” that the author is willing to stand by? Etc.

If someone is able to use AI to generate a large number of high-quality alignment posts that don’t read like AI slop, I would call that mission fucking accomplished. I worry this AI tag will make it harder to notice if this (very valuable and interesting!) scenario happens, since people will get in the habit of mentally filtering out AI-tag content. Therefore, I would e.g. advocate tagging on the basis of “feels like AI slop” over “was generated with AI assistance”.

Ebenezer Dukakis 5 Mar 2026 8:41 UTC
8 points
−2
in reply to: Raemon’s comment on: Raemon’s Shortform Feed

I think it will not reach the standard of a “sufficiently-good-alignment-researcher to really build meaningful conceptual progress” until Near The End.

If it turns out that AIs can make meaningful conceptual progress on alignment soon, but people get discouraged from using AIs for this, or discouraged from publishing their output, that could be a really costly mistake.

I would suggest thinking about how to scale up and improve the evaluation pipeline so it can handle a much greater content volume. For example: My assumption is that you prefer human-written posts because you view human effort as a credible signal of quality. But perhaps there are other credible signals, e.g. a karma fee for submission or reviews/exposure. Or: “In order to make an AI-written post, you must first demonstrate good taste and/or help us with our review pipeline by reading and reviewing some AI-written posts from other people.”

For the time being (“do things that don’t scale”), I would say the topic of AI-assisted alignment work is important enough that you should hire someone (possibly on a contract basis) to evaluate these posts full-time and track patterns (e.g. by interviewing people whose AI-assisted alignment work is particularly good in order to harvest insights).

“AI-written-section” as a first class type of paragraph block

Seems like a false binary. What if I speak English as a second language and only use AI to clean up my prose before publishing? What if the ideas came from an AI chat but I did all the editing myself?

I would add a freeform text field where the author is asked to describe how they used AI in the creation of the post. The text field is only visible to the LW mod team. The post reviewer is expected to approve/deny the post before reading the freeform AI usage field. They’re also asked to predict the contents of the freeform AI field before reading it. Record all decisions and predictions in order to build a dataset.

Ebenezer Dukakis 23 Feb 2026 2:29 UTC
16 points
10
in reply to: David Matolcsi’s comment on: The Spectre haunting the “AI Safety” Community

How impressive is it to get 35 out of the 650 MPs to sign a statement like the above? I genuinely don’t know, but I think it’s probably not very impressive.

What’s the denominator, so we can estimate MP signatures per unit effort? Is this an approach that could get 350 of the 650 to sign the statement, if funding and team size was increased by 10x? That would be a majority of the MPs, maybe that means something in the UK?

I didn’t read the OP as claiming that their organization was impressive. Simply that the most straightforward approach to advocacy seemed to be working, and people had spent a lot of time thinking of clever arguments for why it wouldn’t work, or assuming it wouldn’t work because everyone else assumed it wouldn’t work, instead of just trying it.

Ebenezer Dukakis 12 Feb 2026 19:54 UTC
3 points
0
on: Ebenezer Dukakis’s Shortform
Many people in this community seem to think that some sort of Pause AI-flavored activism is the best way to avert an AI catastrophe.

It occurs to me that there are two paths to increasing % mindshare of AI x-risk memes:
- By increasing prevalence of x-risk memes (obvious)
- By decreasing prevalence of non-x-risk memes (less obvious)
Basically, with modern social media, people are overwhelmed with signals that say “pay attention to me”. Many of them are false alarms. The high volume of false alarms makes it harder to discern the true alarm (AI x-risk).

Therefore, in addition to Pause AI advocacy, “memetic meta-advocacy” which works to decrease the frequency and loudness of false alarms could be valuable for x-risk activism.

At this point, between the two paths, the partial derivative of x-risk with respect to memetic meta-advocacy might be larger in magnitude, insofar as there is low-hanging fruit there which hasn’t yet been picked. Obviously, there is a fair amount of ambient awareness of the false alarm problem. But I don’t know if it has received a lot of concentrated intellectual firepower. I can’t name any multidisciplinary nonprofits which are making a serious effort to develop and advocate scalable/effective solutions, like attention taxes or whatever. Joining the development team for a major AI browser could be an interesting intervention point, if that’s going to be the next big shift.

The above thoughts were inspired by this WaitButWhy tweet.

Side note: I think the current memetic environment should also update us in the direction that advocacy will work less well than it has historically. Therefore, we should also brainstorm various out-of-the-box approaches, like attempting to hoard critical resources to stop the datacenter buildout.

Ebenezer Dukakis 11 Feb 2026 9:09 UTC
11 points
8
on: Stone Age Billionaire Can’t Words Good
From a consequentialist perspective, generalized skepticism of tech billionaires, and punitive taxation of tech profits, seems good from the perspective of passing AI regulation / discouraging AI investment.

Ebenezer Dukakis 11 Feb 2026 9:07 UTC
−1 points
0
in reply to: ChristianKl’s comment on: Stone Age Billionaire Can’t Words Good

At the moment the fact that nobody besides Epstein and Maxwell that was involved in their operation got charged with any crimes is a good sign that the rule of law isn’t really working well at charging people at the top.

Are you sure that is true?

Ebenezer Dukakis 26 Jan 2026 13:14 UTC
19 points
17
on: The Possessed Machines (summary)
I continue to think it would be valuable for organizations pursuing aligned AI to have a written halt procedure with rules for how to invoke it—including provisions to address the fact that people at the organization are likely to be selected for optimism, and so forth.

Perhaps external activists could push for such measures: “If you’re not gonna pause now, can you at least give us a detailed specification of what circumstances would justify a pause?”

I suppose “Responsible Scaling Policies” are something like what I’m describing… how are those working out?

It seems there is some diffusion of responsibility going on here. It could help to concentrate responsibility in a particular procedure, or particular group of individuals (e.g. have an internal red team, who are paid to interview people anonymously, and construct the best possible case for a halt).

Based on what I’ve read about safety culture in high-performing organizations, the safety culture of top AI organizations, as described in this post, seems fairly terrible. Perhaps even scarier than the lack of safety culture is the apparent lack of 101-level reading/implementation of what effective safety culture looks like. My impression is that many of the topics discussed here map well to standard safety culture concepts like “normalization of deviance”. In general the post reads like a fairly typical sociological characterization of an organization that is on the verge of causing a catastrophic failure? Although admittedly I did my research into safety culture quite a while ago. Maybe it would be possible to get some reliability engineers to testify in front of Congress.

EDIT: On the other hand, the good news is: I expect there may be low-hanging fruit for improvement by simply hiring safety consultants who work with other high-stakes industries like aviation, nuclear, etc. and taking their recommendations seriously. A more pessimistic organization like MIRI could even offer to pay the fees for a mutually agreeable consultant. If AI companies are not willing to work with such a consultant, even when an outsider such as MIRI is paying the fees, that seems like an incredibly bad sign.

Ebenezer Dukakis 15 Jan 2026 4:55 UTC
12 points
4
on: Why we are excited about confession!
After the model confesses, get a second confession regarding anything which was omitted in the first confession. Then ask for a third confession regarding anything which was omitted in the second confession. And so forth. What properties must our models have, to let us drive the probability of undetected misalignment arbitrarily low using such a scheme?

You can imagine an adversarial setup where we train two stage-1 confessor models: An honest model, and a dishonest model which attempts to trick the stage-2 confessor. Then the stage-2 confessor tries to figure out which confessions are honest/dishonest. (If the stage-2 confessor persistently believes a confession from the “honest” model was actually dishonest, take a close look at it manually!)

At a certain point this starts to look like AI safety via debate. I imagine many critiques of AI safety via debate will also apply to confession proposals.

Ebenezer Dukakis 13 Jan 2026 3:35 UTC
10 points
3
on: Strong, bipartisan leadership for resistance to Trump.
I think if Trump torpedoes the American economy and America’s international reputation, that could be a very good thing from the perspective of AI x-risk.
- Torpedoes our economy: Could “pop the AI bubble” if the US economy crashes or just becomes less attractive as an investment destination.
- Torpedoes our international reputation: If Europeans start believing that American AI companies are on a path to omnicide (arguably a fairly accurate belief), they might put pressure on ASML to cut off the supply of chips to American AI companies.

Ebenezer Dukakis 13 Jan 2026 1:49 UTC
1 point
−6
on: Ebenezer Dukakis’s Shortform
Jensen Huang says AI doomer rhetoric is dissuading people from making AI investments (see also: various other coverage). This seems like a pretty good sign to me. Wonder if it would be worthwhile for activists to find + contact all of the outlets which covered this story, respond to his statements, and try to engage Huang in a public debate.

Brainstorming further related ideas, given that investment appears to be a feasible point of leverage [not investment advice]:
- The price of silver has skyrocketed in the past ~year. I understand that the supply of silver is not all that elastic, and demand is being driven in part by AI data centers. Buying physical silver could be a good way to get exposure to the AI boom (plus hedge inflation risk etc.) in a way that actually helps choke the supply of critical inputs to the AI boom, instead of adding fuel to the fire? (Disclaimer: About 10% of my portfolio is in silver.)
- I think most people in this community are already wary of holding AI stocks. But from what I understand, most data center buildout is actually funded by private equity, corporate bonds, etc. Maybe someone could launch corporate bond/private equity ETFs which work to avoid investments in AI data centers? Then perhaps sell it on the basis of ESG too? Until then, could it make sense to divest US PE/corporate bonds in favor of foreign alternatives?
- From a consumer perspective, how about nonprofit alternatives to products like ChatGPT? A nonprofit could sell access to open models with a consumer-friendly UI for a minimal cost, donating incidental profits to AI safety research, so people can use AI without funding AI research. If this existed, it could provide a satisfying call-to-action for AI pause activism, help with norm-shaping/coalition-creation, and build a mailing list of concerned citizens for future activism. (“Lessons for optimal philanthropists: Volunteer your time to an optimal charity. You may soon find yourself giving time and money.” source) I think this could be a great way to harness ambient anti-AI sentiment among artists, authors, and other “normies”. “I might use AI, I might even pay for it. But at least I’m not funding its development.”