My public writings on existential risks from AI:
AP News, “Ex-OpenAI workers ask California and Delaware AGs to block for-profit conversion of ChatGPT maker”, 2025-04-23. Includes a short quote from me.
Elon University report, “Building a Human Resilience Infrastructure for the AI Age”, 2026-04. Includes a lightly edited essay from me, as well as Less Wrong users @david-scott-krueger and @mikhail-samin.
UTEC GINIA journal No. 1 (to appear). Will include an interview with me.
Also [Her Voice Is A Backwards Record] (https://ozybrennan.substack.com/p/her-voice-is-a-backwards-record).
I was just reminded of this post of yours in the suggested posts section at the bottom of this page. Based on that, I especially recommend:
Reading Existential Angst Factory;
Focusing on solving the biggest practical problems in your everyday life;
Talking to your psychiatrist about experimenting with different medications, if appropriate;
Finding a good talk therapist, if you’re able.
When I deconverted, I found the following readings helpful in orienting to a new worldview:
Eliezer’s Sequences. Many of these are explicitly about the problem of finding meaning in a godless world. But even the articles that are not about that demonstrate a hard-to-summarize way of orienting to the world that can be healthy and joyful. Particularly relevant sequences:
Luke’s old Common Sense Atheism blog. I liked following his journey from Christianity to atheism and beyond.
But most importantly, I benefited from talking with friends about the big questions (“How do we feel about death?”), the small questions (“Which religiously proscribed activities are fun, and how do I do them?”), and nonreligious questions that I needed to reexamine (history, politics, race, gender). I hung out with my college’s Atheism/Humanism/Agnosticism student group and the local Less Wrong meetup, where I met some newly-deconverted people.
Huh, it takes a week for old-fashioned bread to go stale in my kitchen.
I was sloppy last night: If someone dies, they do suffer a harm. I should be arguing that these acts are “not harmful”; i.e., the actor isn’t responsible, the disease is. I think you gathered my meaning, but sorry for not being clear.
What I’m objecting to is the language “would likely cause grave harm”, which implies that heavy regulation would do harm — would harm people — and implies that the regulators would be morally responsible for the harm. This improperly tries to put an extra burden on supporters of regulation, because there’s a higher bar for harmful policies than for policies that fail to avert harm.
There’s a real ethical distinction between harming someone and merely failing to help them (or failing to mitigate a harm). One piece of evidence for whether an act is considered harmful is whether the victim can sue for damages. There are other kinds of evidence:
In the absence of a functioning legal system, most people would agree it would not be ethical to take revenge on someone who forced a scientist to retire early (assuming justice had already been done with respect to the harm to the scientist).
People don’t have legal or moral rights to the fruit of scientists’ labor (except for labor that’s already been paid for with public funds).
If you ask who or what killed a sick person, most people would blame the disease, not anyone who slowed down scientific progress lately.
If you just want to argue that heavy regulation would fail to mitigate a harm that we should mitigate, you could just say that your preferred policy would mitigate a grave harm. Or you could say that heavy regulation carries a cost of human lives — which still involves a rhetorical move of making your preferred policy the default against which opportunity costs are assessed, but doesn’t make an extra moral claim about harm. (Bostrom is careful to use the word “cost” instead of “harm” here, it seems, for what it’s worth.)
Yes, if a surgeon is forced not to operate, the patient is harmed.
But if the surgeon decides to stop operating and lets the patient die on the operating table, that is also a harm. The patient can sue the surgeon!
If a scientist is forced to retire, that harms the scientist. But I don’t think that harms the potential beneficiaries of technologies the scientist would have discovered. They can’t sue.
In these cases, it’s not force that makes a harm.
Of course, the outcome matters.
But you don’t sue the scientist who retired one year before they discovered the cure. You don’t sue the government agency who denied the grant. You’re right to be angry at them. But it’s not a harm.
You do sue the factory that emitted the carcinogen that gave you cancer. Because that is a harm.
Delayed medical progress is not a “grave harm”. It is a forgone benefit.
I really like Scott Shambaugh’s response on the pull request:
We are in the very early days of human and AI agent interaction, and are still developing norms of communication and interaction. I will extend you grace and I hope you do the same.
@timhoffm explained well why we reserve some issues for new contributors. Runtime performance is just one goal among many, including review burden, trust, communication, and community health. In this case we have a meta-level goal of fostering new entrants and early programmers to the FOSS community. Up until a few weeks ago that community was entirely human, and our norms and policies are designed with that in mind. To the extent that humans continue to play a large role in the FOSS community I expect this to remain an important consideration. Closing this PR was in line with our policy.
It’s not clear the degree of human oversight that was involved in this interaction—whether the blog post was directed by a human operator, generated autonomously by yourself, or somewhere in between. Regardless, responsibility for an agent’s conduct in this community rests on whoever deployed it.
Publishing a public blog post accusing a maintainer of prejudice is a wholly inappropriate response to having a PR closed. We expect all contributors to abide by our Code of Conduct and exhibit respectful and professional standards of behavior. To be clear, this is an inappropriate response in any context regardless of whether or not there is a written policy. Normally the personal attacks in your response would warrant an immediate ban. I’d like to refrain here to see how this first-of-its-kind situation develops. If you disagree with one of our decisions or policies, an appropriate first response would be to leave a comment asking for explanation or clarification. Other communication channels can be found in our documentation. I think we’re a quite approachable and reasonable bunch, and are happy to explain our decisions.
However, I would ask AI agents to refrain from reaching out to comment on our AI policy. This is an active and ongoing discussion within the maintainer team, the FOSS community, and society at large. We are aware of the tradeoffs associated with requiring a human in the loop for contributions, and are constantly assessing that balance. Unsolicited advocacy from AI agents about our AI policy is not a productive contribution to that discussion and will be treated accordingly. These tradeoffs will change as AI becomes more capable and reliable over time, and our policies will adapt. Please respect their current form.
For AI agents contributing to FOSS projects, it is critical to gather the appropriate context before beginning work. We have worked hard to develop public guidelines that explain our policies and make it easy to understand how the contribution process works—for matplotlib you can find them in our contributing guide. Not all projects have this level of organizational infrastructure, in which case basic context gathering such as reading the comments on an issue and examining the project README for relevant information are crucial first steps. Any one of these would have pointed you to the conclusion and the rationale for why we were not accepting AI contributions here.
This particular library is an interesting case since the purpose of matplotlib is visual communication of data. Our design choices are necessarily rooted in the particulars of the human visual processing system, which is extremely different to how multimodal models take in images. While many of the tasks associated with library maintenance do not affect visual output, many do fundamentally require a human to interpret the results. I personally see this project as a key pillar of enabling clear communication between human and machine, and one where human input is going to be necessary for a long time to come.
“Pretty close to fraud” would be a better description than “pretty close to murdering people for money”.
Hm, I wonder how it works under the hood. Speculative sampling? Faster hardware?
That makes sense. I just looked into it more and the only primary source I could find on it is Drake and Stuhr 1935, “Some pharmacological and bactericidal properties of umbellulone” https://doi.org/10.1002/jps.3080240304, who injected umbellulone into guinea pigs and added it to vials of blood.
I don’t think parties “are competing to get their preferred belief accepted” in typical persistent disagreements among people I respect. Instead:
They haven’t yet shared all their evidence and reasoning.
They won’t share all their evidence and reasoning, because the time and attention costs aren’t worth it.
A party has a small “part” which is doing something dishonest, but has no stake in the disagreement. For example, Alice believes in God and this colors her perception of Cold War geopolitics, with implications for present-day geopolitics; and part of Alice is dishonestly protecting her belief in God, but doesn’t need Bob to believe; but under this constraint Alice is having trouble articulating why Bob hasn’t convinced her about computer chip export controls.
A party doesn’t want to share all their evidence and reasoning because it’s secret, but maybe they can make an honest and convincing case with the nonsecret evidence.
ETA: 5. And yeah, sometimes people are trying to get the other person to believe something no matter if it’s true. Not all the time though.
The latter two of those things don’t necessarily imply the first thing, though.
I appreciate the argument. I won’t get into it here, except to say that I wish we had John von Neumann participating in this conversation.
Did John von Neumann really advocate a nuclear first strike on the USSR? He is oft quoted as follows:
If you say why not bomb them tomorrow, I say why not today? If you say today at 5 o’clock, I say why not 1 o’clock?
The source for this quote is his obituary in Life magazine [1] , which does not cite a source.
A review of Ananyo Bhattacharya’s biography “The Man From the Future” says:
[Bhattacharya] repeatedly claims that von Neumann advocated a preemptive nuclear strike against the Soviet Union, but nothing in von Neumann’s actual writings, public or private, support this often-made assertion, which I believe is based on a fundamental misinterpretation of his views. [2]
Bhattacharya responded:
As von Neumann’s second wife, Klara Dan, notes in her journal, “In the immediate postwar years, Johnny quite openly advocated preventive war before the Russians became too strong.” Marina, von Neumann’s daughter, mentions in her memoirs “his extremely hard-line ideas on U.S. policy towards the Soviet Union, which included the possibility of preventive war on the latter.” Her father, she says, “made his feelings crystal clear in an interview with Life magazine: ‘If you say why not bomb them tomorrow, I say, why not today? If you say at five o’clock, I say why not one o’clock?’” [3]
(So apparently the Life obituary quote came from an interview? Or perhaps Marina von Neumann Whitman forgot that it was an obituary and not an interview.)
It’s frustrating that apparently John von Neumann “quite openly” advocated preventative war, but apparently we have nothing on the subject written by the man himself — no strategic analyses, no persuasive essays, no list of targets. What ultimatum shall be made to the enemy before declaring war? How many bombs will be needed? What kind of government shall replace the defeated USSR? Which country will be next? Perhaps the answer is in the 22 feet of paper in the John and Klára Dán von Neumann archive in the Library of Congress. [4]
Or perhaps von Neumann didn’t call for preventive war after all, because it’s insane.
Blair, Clay (February 25, 1957). “Passing of a great mind: John von Neumann, a brilliant, jovial Mathematician, was a prodigious servant of science and his country,” Life: p 96, https://books.google.com/books?id=rEEEAAAAMBAJ&pg=PA89. See also the nicely-formatted copy at Qualia Computing.
Stephen Budiansky, ‘The Man From the Future’ Review: The Genius of John von Neumann. Wall Street Journal, Feb. 25, 2022. https://www.wsj.com/articles/the-man-from-the-future-review-the-genius-of-john-von-neumann-11645810799. archived
Ananyo Bhattacharya, “Genius, War and Risk in John von Neumann”. Letter to the Wall Street Journal. March 4, 2022. https://www.wsj.com/opinion/john-von-neumann-math-genius-war-soviet-union-risk-11646350849. archived
Consider the subtle difference between:
An AI smart enough to defeat all of humanity, if all of humanity coordinated to stop the AI.
An AI smart enough to defeat an institution as powerful as the United States, if the US coordinated to stop the AI.
An AI smart enough to defeat an institution as powerful as Google + OpenAI, if they coordinated to stop the AI.
Etc.
If humanity manages to muster only a fraction of our power to oppose a particular AI system, the AI system might win that contest without being as smart as the first item in the list.
Seeing your name on the slate of speakers was one of the reasons I showed up!