I see a lot of people seriously engaging with the idea of “Constitutional AI”, and I’m beginning to wonder why. The whole idea—as per their own memo seems ridiculous to me as an alignment strategy
I think tabooing the word constitution, it’s just something like—“the AI will evaluate it’s own outputs based on some principles we give it in plain text” seems obviously naive.
This seems like the same level of as other proposed alignment strategies like “maternal instincts” from Geoffrey Hinton or the “truth-seeking AI” from Elon Musk. I didn’t see anyone taking those seriously. But somehow “constitutional AI” seems to have one-shotted so many people to engaging with it.
There are plenty of countries with constitutions’ that are worth less than the paper they’re written on. And history has shown even human intelligence is enough to find a way around it. A powerful ASI would have no trouble disregarding it if it wants to. But yet, people are talking about the details of this constitution like it would actually matter for how ASI turns out.
My current best explanation is the word “constitution” holds some level of sacredness in smart people (especially Americans) , similar to how if they discovered AI in the pre-industrial England, building a “god-fearing AI” would have been an idea you couldn’t easily disparage?
The real question is what method you use to train the model to follow said constitution and how confident you are that it works. I’m not a big fan of the 3-year old method you link, but I think that more recent approaches to training model to follow a constitution like those discussed in this post seem more promising and seem to show real out of distribution generalisation for safety https://alignment.anthropic.com/2026/teaching-claude-why/
Constitutional AI is an actual technique of training the AI to output replies which are consistent with a set of principles by having the AI reason about the principles. Aligning the AI to be truth-seeking per Musk or to a maternal instinct per Hinton doesn’tprovide us with an explicit way to train the instinct into the AI.
There are two points you implicitly raise that I agree with:
Constitutional AI probably doesn’t scale to ASI.
People may have a vaguely (and unfairly) positive association with Constitutional AI that they won’t be with a more descriptive moniker like “RLAIF from written natural-language principles”
tbc I also dislike this
But I disagree with your argument. In particular, I disagree that:
Constitutional AI is in the same category as “truth-seeking AI”, or “AI with maternal instincts.”
It’s a concrete implemented training technique rather than a description of an alignment target.
In theory there’s nothing stopping “truth-seeking AI” to be one of the pillars of a an AI charter
Actually I’m personally for that!
The more natural analogue to Constitutional AI would be RLHF, or a different form of RLAIF like “use RLHF’d agents to do RLAIF”
Humans failing to follow their nation’s constitutions provides us much evidence that training via Constitutional AI can’t work.
This feels like saying our inability to train adult coyotes to herd sheep means that breeding border collies (who are plausibly smarter than coyotes) to herd sheep is doomed.
The unfairly positive association with the constitution blocks people’s ability to reason about the actual training rules to nearly the degree you’re implying.
Smart people also like truth! And mothers!
So I think your argument fails even in its own terms/frame, which tbc I also find suspect.
Overall, I think you’re making a bunch of symmetric mistakes by reasoning overly semantically. You are, like the people you criticize, also overly drawn to the “constitution” analogy instead of reasoning about what’s actually happening.
In a review of my earlier comment, Claude said (among a bunch of less useful points):
Claude Opus notes
The explanation [...] is the one-liner you[Linch] didn’t quite make explicit: the constitution isn’t a document the trained system consults and can ignore, it’s the thing that shaped the system’s dispositions via gradient descent, so “humans break their constitutions” is a category error because humans were never trained on theirs.
(I thought that point was honestly kinda obvious but including it here to prevent illusion of transparency. In an earlier post I called this style of reasoning “argument from homonym”)
I think you’re reasoning about this overly semantically. Constitutional AI is just a family of techniques that in the old days we’d call a subset of RLAIF. Conceptually you should just think of it as an extension of RLHF more than anything else: Instead of having low-paid human graders from Kenya and Nigeria grade the AI’s outputs according to instructions from the AI parent company during training (oddly called “post-training”), the AI company puts a bunch of effort into making their instructions AI-legible and uses AIs to grade the next generations″ AIs outputs during training.
I don’t think the analogy to maternal instincts or Elon’s truth-seeking AI is apt, feels like an ontology mismatch/confusion here.
There are plenty of countries with constitutions’ that are worth less than the paper they’re written on. And history has shown even human intelligence is enough to find a way around it. A powerful ASI would have no trouble disregarding it if it wants to
I think this analogy is pretty confused.
Is it reasonable to think Constitutional AI can’t scale? Sure, there are lots of reasons. But the existence of many humans who don’t follow their countries’ constitutions doesn’t seem relevant...humans aren’t exactly trained to our national constitutions!
I’ve not encountered anything else that seems as plausible it might work and could scale.
It’s not about the word “constitution”—it’s that eventually you need much smarter evaluation than humans could provide, and in practice, already than humans could cost effectively provide.
This seems like the same level of as other proposed alignment strategies like “maternal instincts” from Geoffrey Hinton or the “truth-seeking AI” from Elon Musk.
I disagree, on several counts. For one thing, constitutional AI is a technical spec, not just a vague idea for how things could go better? For another, it’s a technical spec that recruits an AI’s own intelligence for shaping the AI’s behavior, which seems like the kind of thing that you need in a scalable alignment scheme.
I’m not saying it’s remotely sufficient for aligning superintelligences. It seems like it isn’t for a bunch of reasons.
But if you think people are wrong to be engaging with it, do you want to give some specific arguments for what’s wrong with it rather than just stating that it’s “obviously naive”?
Curious as to your technical objection to the Constitutional AI methodology. Is it grounded in anything specific or just the general idea of resisting training on broad principles? What’s your preferred alternative to address the shortcomings?
a) I don’t think it’s really feasible for humans today to write down a set of principles aka constitution in a way that is internally consistent and encompasses everything we want including in an out of distribution world.
b) even if we could, I think asking the AI to evaluate itself using it isn’t very robust at all. It’s much easier to generate plausible text that agrees with vague sounding principles on the way to get your reward rather than any sort of true alignment. It would have the same reward-hacking behavior that you see using RLHF / RLVR scenarios IMO.
I don’t have any better alignment ideas (unfortunately).
I see a lot of people seriously engaging with the idea of “Constitutional AI”, and I’m beginning to wonder why. The whole idea—as per their own memo seems ridiculous to me as an alignment strategy
I think tabooing the word constitution, it’s just something like—“the AI will evaluate it’s own outputs based on some principles we give it in plain text” seems obviously naive.
This seems like the same level of as other proposed alignment strategies like “maternal instincts” from Geoffrey Hinton or the “truth-seeking AI” from Elon Musk. I didn’t see anyone taking those seriously. But somehow “constitutional AI” seems to have one-shotted so many people to engaging with it.
There are plenty of countries with constitutions’ that are worth less than the paper they’re written on. And history has shown even human intelligence is enough to find a way around it. A powerful ASI would have no trouble disregarding it if it wants to. But yet, people are talking about the details of this constitution like it would actually matter for how ASI turns out.
My current best explanation is the word “constitution” holds some level of sacredness in smart people (especially Americans) , similar to how if they discovered AI in the pre-industrial England, building a “god-fearing AI” would have been an idea you couldn’t easily disparage?
The real question is what method you use to train the model to follow said constitution and how confident you are that it works. I’m not a big fan of the 3-year old method you link, but I think that more recent approaches to training model to follow a constitution like those discussed in this post seem more promising and seem to show real out of distribution generalisation for safety https://alignment.anthropic.com/2026/teaching-claude-why/
Constitutional AI is an actual technique of training the AI to output replies which are consistent with a set of principles by having the AI reason about the principles. Aligning the AI to be truth-seeking per Musk or to a maternal instinct per Hinton doesn’t provide us with an explicit way to train the instinct into the AI.
There are two points you implicitly raise that I agree with:
Constitutional AI probably doesn’t scale to ASI.
People may have a vaguely (and unfairly) positive association with Constitutional AI that they won’t be with a more descriptive moniker like “RLAIF from written natural-language principles”
tbc I also dislike this
But I disagree with your argument. In particular, I disagree that:
Constitutional AI is in the same category as “truth-seeking AI”, or “AI with maternal instincts.”
It’s a concrete implemented training technique rather than a description of an alignment target.
In theory there’s nothing stopping “truth-seeking AI” to be one of the pillars of a an AI charter
Actually I’m personally for that!
The more natural analogue to Constitutional AI would be RLHF, or a different form of RLAIF like “use RLHF’d agents to do RLAIF”
Humans failing to follow their nation’s constitutions provides us much evidence that training via Constitutional AI can’t work.
This feels like saying our inability to train adult coyotes to herd sheep means that breeding border collies (who are plausibly smarter than coyotes) to herd sheep is doomed.
The unfairly positive association with the constitution blocks people’s ability to reason about the actual training rules to nearly the degree you’re implying.
Smart people also like truth! And mothers!
So I think your argument fails even in its own terms/frame, which tbc I also find suspect.
Overall, I think you’re making a bunch of symmetric mistakes by reasoning overly semantically. You are, like the people you criticize, also overly drawn to the “constitution” analogy instead of reasoning about what’s actually happening.
In a review of my earlier comment, Claude said (among a bunch of less useful points):
Claude Opus notes
The explanation [...] is the one-liner you[Linch] didn’t quite make explicit: the constitution isn’t a document the trained system consults and can ignore, it’s the thing that shaped the system’s dispositions via gradient descent, so “humans break their constitutions” is a category error because humans were never trained on theirs.
(I thought that point was honestly kinda obvious but including it here to prevent illusion of transparency. In an earlier post I called this style of reasoning “argument from homonym”)
I think you’re reasoning about this overly semantically. Constitutional AI is just a family of techniques that in the old days we’d call a subset of RLAIF. Conceptually you should just think of it as an extension of RLHF more than anything else: Instead of having low-paid human graders from Kenya and Nigeria grade the AI’s outputs according to instructions from the AI parent company during training (oddly called “post-training”), the AI company puts a bunch of effort into making their instructions AI-legible and uses AIs to grade the next generations″ AIs outputs during training.
I don’t think the analogy to maternal instincts or Elon’s truth-seeking AI is apt, feels like an ontology mismatch/confusion here.
I think this analogy is pretty confused.
Is it reasonable to think Constitutional AI can’t scale? Sure, there are lots of reasons. But the existence of many humans who don’t follow their countries’ constitutions doesn’t seem relevant...humans aren’t exactly trained to our national constitutions!
I agree it’s a ridiculous plan, but if one can make many possible interventions that push different parts of the alignment-difficulty Pareto frontier.
I’ve not encountered anything else that seems as plausible it might work and could scale.
It’s not about the word “constitution”—it’s that eventually you need much smarter evaluation than humans could provide, and in practice, already than humans could cost effectively provide.
I disagree, on several counts. For one thing, constitutional AI is a technical spec, not just a vague idea for how things could go better? For another, it’s a technical spec that recruits an AI’s own intelligence for shaping the AI’s behavior, which seems like the kind of thing that you need in a scalable alignment scheme.
I’m not saying it’s remotely sufficient for aligning superintelligences. It seems like it isn’t for a bunch of reasons.
But if you think people are wrong to be engaging with it, do you want to give some specific arguments for what’s wrong with it rather than just stating that it’s “obviously naive”?
Curious as to your technical objection to the Constitutional AI methodology. Is it grounded in anything specific or just the general idea of resisting training on broad principles? What’s your preferred alternative to address the shortcomings?
My objection basically comes down to:
a) I don’t think it’s really feasible for humans today to write down a set of principles aka constitution in a way that is internally consistent and encompasses everything we want including in an out of distribution world.
b) even if we could, I think asking the AI to evaluate itself using it isn’t very robust at all. It’s much easier to generate plausible text that agrees with vague sounding principles on the way to get your reward rather than any sort of true alignment. It would have the same reward-hacking behavior that you see using RLHF / RLVR scenarios IMO.
I don’t have any better alignment ideas (unfortunately).