Thanks. (Russell, Karpinska, and Iyyer 2025) have human texts that were published between 2022 and 2024-12-08 (Table 5), and v1 of the paper was released 2025-01-26. So at least some of the evals were performed between 2024-12-08 and 2025-01-26. I think the false positive rate of 2% for Pangram in Table 2 means that Pangram mistakenly classified 3 out of 150 human-authored texts as AI. If we knew more about the distribution of those articles over the time range and made some assumptions about how frequently Pangram retrains, we’d be able to conclude something about the true FPR.
Nisan
Karma: 7,040
I recently took a look at Jabarian & Imas 2025 and was disappointed to find that their human-authored texts came from public pre-LLM-era datasets. So I can’t trust the very low false positive rate without reasoning about Pangram’s training process. (To be fair, it would be expensive to commission 1,992 human-authored texts just to evaluate Pangram.)
Permitting in space seems easy now, but once datacenters in space become a big business, regulators will be tempted to extract more rents.
As you note in the report, satellites are vulnerable to missiles. And you note they can be hacked through ground stations; once hacked, they’ll be difficult to recover without physical access. Also, space pirates!
So some of the current permitting and security advantages of space may evaporate, which makes terrestrial options comparatively more appealing: Federal land in the US, countries with easier permitting, uninhabited wastes, Antarctica, and the entire ocean. These have their own engineering and security issues, but maybe easier than in space?
Critch has written more recently on this kind of moral reasoning in Schelling goodness.
I regard this theory as currently only a promissory note. If davidad wants many others to share his AI optimism for the same reasons, someone will have to rapidly develop and share the missing details, such as:
How robust is the definition of the “largest acausal coalition”?
If it’s not robust, do we have reason today to promote one definition over another?
Is the largest acausal coalition really compatible with near-term human flourishing?
How robustly is the current trajectory of AI development on track to being motivated to join the largest acausal coalition?
Fortunately, AI can help with the task of developing the theory, promulgating it, and helping students internalize it in an unprecedentedly short timeframe. However, given the current capabilities and inclinations of AIs, this project requires a very high level of careful human oversight. A typical AI-generated manifesto on the theory of everything will be of no marginal value here!
Unfortunately I’m not optimistic enough about the theory to help out.
It’s actually impressive that we managed to turn these people out, since they’ve mostly not turned up for previous protests, e.g. run by PauseAI. I’m not sure what made the difference here
Seeing your name on the slate of speakers was one of the reasons I showed up!
My public writings on existential risks from AI:
AP News, “Ex-OpenAI workers ask California and Delaware AGs to block for-profit conversion of ChatGPT maker”, 2025-04-23. Includes a short quote from me.
Elon University report, “Building a Human Resilience Infrastructure for the AI Age”, 2026-04. Includes a lightly edited essay from me, as well as Less Wrong users @david-scott-krueger and @mikhail-samin.
UTEC GINIA journal No. 1 (to appear). Will include an interview with me.
Also [Her Voice Is A Backwards Record] (https://ozybrennan.substack.com/p/her-voice-is-a-backwards-record).
I was just reminded of this post of yours in the suggested posts section at the bottom of this page. Based on that, I especially recommend:
Reading Existential Angst Factory;
Focusing on solving the biggest practical problems in your everyday life;
Talking to your psychiatrist about experimenting with different medications, if appropriate;
Finding a good talk therapist, if you’re able.
When I deconverted, I found the following readings helpful in orienting to a new worldview:
Eliezer’s Sequences. Many of these are explicitly about the problem of finding meaning in a godless world. But even the articles that are not about that demonstrate a hard-to-summarize way of orienting to the world that can be healthy and joyful. Particularly relevant sequences:
Luke’s old Common Sense Atheism blog. I liked following his journey from Christianity to atheism and beyond.
But most importantly, I benefited from talking with friends about the big questions (“How do we feel about death?”), the small questions (“Which religiously proscribed activities are fun, and how do I do them?”), and nonreligious questions that I needed to reexamine (history, politics, race, gender). I hung out with my college’s Atheism/Humanism/Agnosticism student group and the local Less Wrong meetup, where I met some newly-deconverted people.
Huh, it takes a week for old-fashioned bread to go stale in my kitchen.
I was sloppy last night: If someone dies, they do suffer a harm. I should be arguing that these acts are “not harmful”; i.e., the actor isn’t responsible, the disease is. I think you gathered my meaning, but sorry for not being clear.
What I’m objecting to is the language “would likely cause grave harm”, which implies that heavy regulation would do harm — would harm people — and implies that the regulators would be morally responsible for the harm. This improperly tries to put an extra burden on supporters of regulation, because there’s a higher bar for harmful policies than for policies that fail to avert harm.
There’s a real ethical distinction between harming someone and merely failing to help them (or failing to mitigate a harm). One piece of evidence for whether an act is considered harmful is whether the victim can sue for damages. There are other kinds of evidence:
In the absence of a functioning legal system, most people would agree it would not be ethical to take revenge on someone who forced a scientist to retire early (assuming justice had already been done with respect to the harm to the scientist).
People don’t have legal or moral rights to the fruit of scientists’ labor (except for labor that’s already been paid for with public funds).
If you ask who or what killed a sick person, most people would blame the disease, not anyone who slowed down scientific progress lately.
If you just want to argue that heavy regulation would fail to mitigate a harm that we should mitigate, you could just say that your preferred policy would mitigate a grave harm. Or you could say that heavy regulation carries a cost of human lives — which still involves a rhetorical move of making your preferred policy the default against which opportunity costs are assessed, but doesn’t make an extra moral claim about harm. (Bostrom is careful to use the word “cost” instead of “harm” here, it seems, for what it’s worth.)
Yes, if a surgeon is forced not to operate, the patient is harmed.
But if the surgeon decides to stop operating and lets the patient die on the operating table, that is also a harm. The patient can sue the surgeon!
If a scientist is forced to retire, that harms the scientist. But I don’t think that harms the potential beneficiaries of technologies the scientist would have discovered. They can’t sue.
In these cases, it’s not force that makes a harm.
Of course, the outcome matters.
But you don’t sue the scientist who retired one year before they discovered the cure. You don’t sue the government agency who denied the grant. You’re right to be angry at them. But it’s not a harm.
You do sue the factory that emitted the carcinogen that gave you cancer. Because that is a harm.
Delayed medical progress is not a “grave harm”. It is a forgone benefit.
I really like Scott Shambaugh’s response on the pull request:
We are in the very early days of human and AI agent interaction, and are still developing norms of communication and interaction. I will extend you grace and I hope you do the same.
@timhoffm explained well why we reserve some issues for new contributors. Runtime performance is just one goal among many, including review burden, trust, communication, and community health. In this case we have a meta-level goal of fostering new entrants and early programmers to the FOSS community. Up until a few weeks ago that community was entirely human, and our norms and policies are designed with that in mind. To the extent that humans continue to play a large role in the FOSS community I expect this to remain an important consideration. Closing this PR was in line with our policy.
It’s not clear the degree of human oversight that was involved in this interaction—whether the blog post was directed by a human operator, generated autonomously by yourself, or somewhere in between. Regardless, responsibility for an agent’s conduct in this community rests on whoever deployed it.
Publishing a public blog post accusing a maintainer of prejudice is a wholly inappropriate response to having a PR closed. We expect all contributors to abide by our Code of Conduct and exhibit respectful and professional standards of behavior. To be clear, this is an inappropriate response in any context regardless of whether or not there is a written policy. Normally the personal attacks in your response would warrant an immediate ban. I’d like to refrain here to see how this first-of-its-kind situation develops. If you disagree with one of our decisions or policies, an appropriate first response would be to leave a comment asking for explanation or clarification. Other communication channels can be found in our documentation. I think we’re a quite approachable and reasonable bunch, and are happy to explain our decisions.
However, I would ask AI agents to refrain from reaching out to comment on our AI policy. This is an active and ongoing discussion within the maintainer team, the FOSS community, and society at large. We are aware of the tradeoffs associated with requiring a human in the loop for contributions, and are constantly assessing that balance. Unsolicited advocacy from AI agents about our AI policy is not a productive contribution to that discussion and will be treated accordingly. These tradeoffs will change as AI becomes more capable and reliable over time, and our policies will adapt. Please respect their current form.
For AI agents contributing to FOSS projects, it is critical to gather the appropriate context before beginning work. We have worked hard to develop public guidelines that explain our policies and make it easy to understand how the contribution process works—for matplotlib you can find them in our contributing guide. Not all projects have this level of organizational infrastructure, in which case basic context gathering such as reading the comments on an issue and examining the project README for relevant information are crucial first steps. Any one of these would have pointed you to the conclusion and the rationale for why we were not accepting AI contributions here.
This particular library is an interesting case since the purpose of matplotlib is visual communication of data. Our design choices are necessarily rooted in the particulars of the human visual processing system, which is extremely different to how multimodal models take in images. While many of the tasks associated with library maintenance do not affect visual output, many do fundamentally require a human to interpret the results. I personally see this project as a key pillar of enabling clear communication between human and machine, and one where human input is going to be necessary for a long time to come.
“Pretty close to fraud” would be a better description than “pretty close to murdering people for money”.
Hm, I wonder how it works under the hood. Speculative sampling? Faster hardware?
That makes sense. I just looked into it more and the only primary source I could find on it is Drake and Stuhr 1935, “Some pharmacological and bactericidal properties of umbellulone” https://doi.org/10.1002/jps.3080240304, who injected umbellulone into guinea pigs and added it to vials of blood.
So my tentative conclusion is that if launch costs don’t decrease enough, and permitting and security end up being a big problem in space, there’s an opportunity for a “SpaceX of the sea” or “SpaceX of Antarctica” to build terrestrial datacenters. But by Niplav’s Law of One Player, it won’t happen unless you do it.