A CERN for AI is a distraction; push for an IAEA instead
TL;DR: There are many conceivable versions of a “CERN for AI.” But the version that seems politically realistic (a new catch-up lab) probably would not do much for safety, while the versions that would materially improve safety (e.g., pause + merge of all companies) are probably unrealistic. So I see the CERN idea as a distraction, and not a particularly neglected one. I argue a better path is an international treaty with red lines now, with an IAEA-style verification body next: a sequencing that matches how the EU AI Act, the NPT/IAEA, and the Montreal Protocol actually developed. This is premised on the view that the main bottleneck in AI safety is enforcement and political will, not more R&D.
Two premises underlie the rationale below:
First, the bottleneck in AI safety is political will and the enforcement of best practices, not more R&D. With enough will, we move from Greenblatt’s Plan D toward Plan A, achieving roughly an 80% risk reduction. Of course, the science of AI safety is far from mature, but we are also far from applying the best risk mitigation practices (see also the various other ratings, from SaferAI to FLI’s one) - of course, alignment is not solved, but also, xAI is still releasing Mecha Hitler…
We already have sufficient verification mechanisms to get started. What’s missing is the decision to use them (link).
If you reject either premise, you might reach different conclusions.
The CERN for AI is a distraction
A recurring proposal in AI governance is to build a “CERN for AI”[1]. The CERN pitch is seductive. “Let’s build together!” That’s sexier than “we need to ban.” You can leverage historical analogies (CERN for physics, NASA) and talk about national interest and science. It sounds like the smarter, more sophisticated play.[1]
But I think that there are many problems with it.
What do you even mean by CERN?
Are you asking for:
a) pausing AI development at OpenAI, etc, and on top of this pause, creating a new institution that conducts ALL the frontier development? Let’s be clear: this will never happen unless you explicitly ask for a pause. And by default, the US and US CEOs will push extremely hard against handing off their power in this way[2]. (I’m not even sure this would be desirable in terms of power concentration, and I’m not sure such a structure would be able to do a better job than Anthropic at pushing frontier AI safety R&D). Push for (a) without saying ‘pause’ and you’ll get (b) by default:
b) a new lab that tries to catch up to frontier labs. But this new lab, trying to catch up, is not reducing risks. Also, every state’s attempt at frontier LLMs has been 2-3 generations behind the labs. The comparative advantage of states isn’t racing frontier labs. It’s regulating them. A CERN asks states to do the one thing they’re worst at. A CERN-for-AI in Europe today would most likely look like a new Mistral, and competitors who try to catch up tend to cut corners on safety and lobby against regulation (cf. Mistral).
c) or maybe you want to create a literal CERN, i.e., a pure research center, which would not necessarily create frontier models? But there is already plenty of research that is being ignored by companies. I believe the bottleneck is currently enforcement and binding regulation. A research center without enforcement teeth doesn’t shift industry incentives.
To be honest, if you really do believe we might face catastrophic risks in a few months or years, I think the priority is to stop the hemorrhage[3]. If what you ultimately want is to mitigate AI risks, say it, and don’t play 4D chess.
We should ask for international regulations now. This leads us to:
Red lines now, IAEA for AI next
For context, the IAEA (International Atomic Energy Agency) issues international nuclear safety standards and red lines, supports peer reviews and inspections, and coordinates assistance during nuclear emergencies. These standards are then adopted and enforced through national legislation worldwide. An IAEA for AI would play a similar role for artificial intelligence.
Red lines have something CERN doesn’t: existing momentum.
Red lines are the most widely supported measure by research institutes, think tanks, and independent organizations. By signing the Seoul Frontier AI Safety Commitments, companies agreed to “Set out thresholds at which severe risks posed by a model or system, unless adequately mitigated, would be deemed intolerable.” Granted, OpenAI’s “red line” for recursive AI self-improvement is currently inadequate, but we’re not building from zero, and this is why red lines need to be binding rather than voluntary.
China’s Premier Li Qiang stated that “there should be a red line in AI development, a red line that must not be crossed.” Pope Francis urged nations to adopt “a binding international treaty.”, and Paolo Benanti, the Pope’s AI adviser, called explicitly for “binding international treaties and red lines.”
Even the Executive order signed by Trump in early June frames this as a threshold definition: NSA will define a cyber-capability threshold, classified and shared with developers.
Red lines need an institution: the IAEA model
The final big hesitation while drafting the Global Call for AI Red Lines was whether to ask explicitly for the IAEA for AI. The main reason we didn’t ask for the IAEA in the global call was mostly to generate more support (“IAEA for AI” sounds technocratic and wonky to non-specialists, while red lines are intuitive). But red lines only matter if they can be independently verified: without an independent authority, there is no trust; without trust, no credible verification; and without credible verification, red lines are unlikely to be respected. This is why the strongest arms-control regimes are paired with international verification mechanisms: the Chemical Weapons Convention with the OPCW, and the NPT with the IAEA.
At the India AI Impact Summit, the CEOs of the three leading frontier AI labs each called for international AI oversight. Altman joined Hassabis in calling for an institution modeled on the IAEA. Amodei called for red lines with enforcement mechanisms.[4] The fact that CEOs have recently publicly called for IAEA-style oversight is one of the strongest arguments for the current US administration.
This sequencing – an international agreement with red lines first, institution second – mirrors how international governance actually works. The EU AI Act passed without every technical threshold defined; the AI Office was established afterward; specific thresholds are currently being defined, by technical consortiums working with the AI Office. Same pattern from the Vienna Convention to the Montreal Protocol, with detailed control measures strengthened gradually through expert-led review. Political agreement creates the conditions for technical work to happen inside the governance process, not before it.
Thanks to Arthur Grimonpont and Epiphanie Gedeon for helpful suggestions and feedback on this memo.
- ^
Even Demis Hassabis, back in the day, said: “At some point in the future, we’ll need a CERN for AGI for international coordination on safety research.”
- ^
If a credible pause-plus-co-development package were actually on the table (something like the AI-2027 slowdown scenario, with DeepMind, Anthropic and OpenAI brought into one project, even though they cannot hold hands at the Delhi Summit), I’d be happy to back option (a). I don’t expect that scenario to materialize. CEOs of frontier companies have way too much ego for this. Even Mistral, which is already generations behind, would resist being folded into a frontier consortium tomorrow. Merging the different frontier labs is wayyyy harder than creating a new institution. The EU AI Office creation went pretty smoothly when you think about it.
- ^
Many people pushing for a CERN have European sovereignty in mind. To be fair, I think that Europe should wake up to the importance of AI. But there are so many ways to do it in a more effective way:
If what you want is sovereignty, the easiest version is to package open source models that are currently just 4 months behind the frontier—not train frontier models from scratch that are 3 generations behind!
If you want safety: enforce the AI act, and serve as a diplomatic power to get safety on the world scene.
If you want to fund moonshot for alignment, I’m very skeptical this is the most direct route
If you want to strengthen your industry, prepare for physical AI and robotics
- ^
The CEOs of the three leading AI companies have each publicly called for international oversight. Dario Amodei said he could imagine a worldwide treaty with enforcement mechanisms. Sam Altman called for “urgent global regulation on AI”, and for an equivalent of the International Atomic Energy Agency for international coordination on AI. Demis Hassabis also called for “some kind of equivalent of the IAEA.” For reference, the IAEA issues international nuclear safety standards and red lines, supports peer reviews and inspections, and coordinates assistance during nuclear emergencies. These standards are then adopted and enforced through national legislation worldwide.
I’ve never looked closely at the question of a CERN for AI. I feel like I’ve heard about it from afar, and then the topic mostly disappeared from my bubble, to be replaced by the idea of an IAEA for AI. I assumed that it was simply the initial intuition which, through organically refining itself within the community, had converged from CERN to IAEA for the reasons you mention in your post.
I imagine that what motivated you to write this post is that you somewhat regularly hear people defending the idea of a CERN for AI, so do you know whether the 3 visions a), b) and c) of the CERN for AI that you describe are defended by many people, and in what proportion?
the following reply has been vibe written with Claude.
I hear versions of this regularly, and the three visions aren’t defended in equal proportion. Almost nobody explicitly argues for (a); it’s mostly (b) and (c), usually without distinguishing them.
The Simon Institute recently catalogued 14 prominent proposals and 3 existing projects that have used the “CERN for AI” label, and their conclusion is basically mine — the label points at completely different things depending on who’s using it: a publicly funded counterweight to commercial labs; an international AI safety institute that tests models; a way to secure European compute competitiveness; or a joint project to actually build frontier models. Worth reading that piece directly.
Here’s who’s defending what, sorted by which version they’re actually asking for:
(a) Pause + merge / single global developer
The version I think would help safety — and the one that’s essentially absent from the mainstream because it requires saying “pause” out loud.
A Narrow Path / MAGIC (Miotti, ControlAI; Hausenloy, Miotti & Dennis, 2023) — the only version honest about the pause. MAGIC would be the only institution permitted to develop advanced AI, enforced by a global moratorium on all other advanced development. This is basically (a), stated explicitly — which is why it’s coherent but treated as politically fringe. FLI even files it under “a ‘CERN for AI’”.
(b) A new lab that races to catch up to the frontier
The politically live default. Pitched as sovereignty or trustworthy AI, ends up as a new lab building models — and, absent a pause, structurally 2–3 generations behind.
Bengio & the Blueprint for Multinational Advanced AI Development (Oxford Martin AIGI, 2026) — a new multinational developer, not a pause. Premise: the US controls ~75% of global AI compute, China 15%, the EU 5%, so mid-sized economies face insurmountable barriers to independent frontier development — hence a pooled effort. (I’m a co-author; it’s the serious, well-governed version of (b), but it’s still (b) — a new institution that builds models, not a merge of existing ones.)
CFG — Building CERN for AI (Centre for Future Generations) — by far the most institutionally detailed, and the clearest case of my worry. It’s framed as trustworthy-AI research (c), but a pioneer institution for trustworthy general-purpose AI with a €35 billion, three-year compute budget and sovereign-model ambition is (b) — a frontier lab, whatever the label says. The CFG lineage overlaps with the Europe 2031 authors (Daan Juijn, Alex Petropoulos at Arq) — a coherent cluster pushing sovereignty-through-capacity.
The Frontier AI Initiative — no longer a “wannabe”; it’s real now. France, Germany and the Commission launched it in November 2025 as a non-profit public-private effort for frontier AI development, billed as the best-funded non-profit frontier AI initiative in the world, with access to Europe’s giga-compute. ECFR explicitly frames middle-power collaboration like this as the possible foundation of the longstanding “CERN for AI” proposal. This is (b) with the mask off — and exactly the trajectory Europe 2031 warns about.
von der Leyen / the Commission — proposed a “CERN for AI” in her July 2024 political guidelines, then rebranded existing programs (the Gigafactories) as “akin to” one — which even sympathetic observers say bears little resemblance to what CERN actually was. Officially framed as competitiveness, i.e. (b).
(c) A pure research / safety center (no frontier training)
Where the bottleneck is enforcement, not more research — so a center without teeth doesn’t shift incentives.
aitreaty.org — squarely (c): asks for a collaborative AI safety laboratory akin to CERN, pooling resources for safety research, alongside global compute thresholds and a compliance commission. Note it pairs the CERN-for-safety with an IAEA-style body — which supports my sequencing point, not the standalone-CERN one.
Straddles / ambivalent
Miles Brundage — genuinely ambivalent, and a strong card to play. He wrote a detailed sketch of a version he likes: pooling many countries’ and companies’ resources into a single, possibly decentralized, civilian development effort — i.e. (a)/(b). But elsewhere he lists “CERN for AI” among the AI-policy proposals he finds too vague to confidently judge. So even a proponent concedes the concept has low “policy readiness.”
The long tail: Hassabis’s old “CERN for AGI” line, CAIRNE’s decade-old open letter and “moonshot” framing, and Gary Marcus’s international agency.
On your convergence instinct
I’d push back slightly. It hasn’t converged from CERN to IAEA — the two run in parallel for different audiences. IAEA dominates the safety/coordination bubble you’re in; CERN dominates the European industrial-policy bubble, where it’s better-funded and more politically live than ever (von der Leyen, CFG, the Frontier AI Initiative). From inside our bubble it looks settled; where the money and the Commission are, it very much isn’t. That’s why I still think it’s worth arguing against.
And this is also dramatized in the AI 2027 lineage — Europe 2031 (Arq Foundation) is explicitly modeled on it, and it stages precisely the failure mode I’m worried about: Europe doubling down on sovereignty but forgetting to build leverage.