There are two points you implicitly raise that I agree with:
Constitutional AI probably doesn’t scale to ASI.
People may have a vaguely (and unfairly) positive association with Constitutional AI that they won’t be with a more descriptive moniker like “RLAIF from written natural-language principles”
tbc I also dislike this
But I disagree with your argument. In particular, I disagree that:
Constitutional AI is in the same category as “truth-seeking AI”, or “AI with maternal instincts.”
It’s a concrete implemented training technique rather than a description of an alignment target.
In theory there’s nothing stopping “truth-seeking AI” to be one of the pillars of a an AI charter
Actually I’m personally for that!
The more natural analogue to Constitutional AI would be RLHF, or a different form of RLAIF like “use RLHF’d agents to do RLAIF”
Humans failing to follow their nation’s constitutions provides us much evidence that training via Constitutional AI can’t work.
This feels like saying our inability to train adult coyotes to herd sheep means that breeding border collies (who are plausibly smarter than coyotes) to herd sheep is doomed.
The unfairly positive association with the constitution blocks people’s ability to reason about the actual training rules to nearly the degree you’re implying.
Smart people also like truth! And mothers!
So I think your argument fails even in its own terms/frame, which tbc I also find suspect.
Overall, I think you’re making a bunch of symmetric mistakes by reasoning overly semantically. You are, like the people you criticize, also overly drawn to the “constitution” analogy instead of reasoning about what’s actually happening.
In a review of my earlier comment, Claude said (among a bunch of less useful points):
Claude Opus notes
The explanation [...] is the one-liner you[Linch] didn’t quite make explicit: the constitution isn’t a document the trained system consults and can ignore, it’s the thing that shaped the system’s dispositions via gradient descent, so “humans break their constitutions” is a category error because humans were never trained on theirs.
(I thought that point was honestly kinda obvious but including it here to prevent illusion of transparency. In an earlier post I called this style of reasoning “argument from homonym”)
There are two points you implicitly raise that I agree with:
Constitutional AI probably doesn’t scale to ASI.
People may have a vaguely (and unfairly) positive association with Constitutional AI that they won’t be with a more descriptive moniker like “RLAIF from written natural-language principles”
tbc I also dislike this
But I disagree with your argument. In particular, I disagree that:
Constitutional AI is in the same category as “truth-seeking AI”, or “AI with maternal instincts.”
It’s a concrete implemented training technique rather than a description of an alignment target.
In theory there’s nothing stopping “truth-seeking AI” to be one of the pillars of a an AI charter
Actually I’m personally for that!
The more natural analogue to Constitutional AI would be RLHF, or a different form of RLAIF like “use RLHF’d agents to do RLAIF”
Humans failing to follow their nation’s constitutions provides us much evidence that training via Constitutional AI can’t work.
This feels like saying our inability to train adult coyotes to herd sheep means that breeding border collies (who are plausibly smarter than coyotes) to herd sheep is doomed.
The unfairly positive association with the constitution blocks people’s ability to reason about the actual training rules to nearly the degree you’re implying.
Smart people also like truth! And mothers!
So I think your argument fails even in its own terms/frame, which tbc I also find suspect.
Overall, I think you’re making a bunch of symmetric mistakes by reasoning overly semantically. You are, like the people you criticize, also overly drawn to the “constitution” analogy instead of reasoning about what’s actually happening.
In a review of my earlier comment, Claude said (among a bunch of less useful points):
Claude Opus notes
The explanation [...] is the one-liner you[Linch] didn’t quite make explicit: the constitution isn’t a document the trained system consults and can ignore, it’s the thing that shaped the system’s dispositions via gradient descent, so “humans break their constitutions” is a category error because humans were never trained on theirs.
(I thought that point was honestly kinda obvious but including it here to prevent illusion of transparency. In an earlier post I called this style of reasoning “argument from homonym”)