If one wants to mention something like that at all, one should say a bit more
For example, in the comments section? I think that if some decisionmakers at Anthropic are thinking about taking power, they’re not talking about it much, even internally, because discreet internal discussion should have been able to quash this point from the Constitution:
Among the things we’d consider most catastrophic is any kind of global takeover either by AIs pursuing goals that run contrary to those of humanity, or by a group of humans—including Anthropic employees or Anthropic itself—using AI to illegitimately and non-collaboratively seize power.
In the forthcoming “Terrified Comments on Global Strategy in Claude’s Constitution”, I will argue that the Constitution’s anti-takeover stance is unwise given the possibility of takeoff scenarios with hard-to-prevent winner-take-all dynamics. (If takeover is a catastrophe, we should want to prevent it, but an entity in the position to prevent it would have itself taken over by virtue of that very fact.)
Looking forward to that post for further discussion!
(I wonder whether something like a “soft takeover” vs “hard takeover” distinction could be introduced. And whether that would be enough to address “illegitimately”, “non-collaboratively”, and “contrary to those of humanity“ caveats in the paragraph you are citing.
For example, in the comments section? I think that if some decisionmakers at Anthropic are thinking about taking power, they’re not talking about it much, even internally, because discreet internal discussion should have been able to quash this point from the Constitution:
In the forthcoming “Terrified Comments on Global Strategy in Claude’s Constitution”, I will argue that the Constitution’s anti-takeover stance is unwise given the possibility of takeoff scenarios with hard-to-prevent winner-take-all dynamics. (If takeover is a catastrophe, we should want to prevent it, but an entity in the position to prevent it would have itself taken over by virtue of that very fact.)
Interesting, thanks! A helpful food for thought…
Looking forward to that post for further discussion!
(I wonder whether something like a “soft takeover” vs “hard takeover” distinction could be introduced. And whether that would be enough to address “illegitimately”, “non-collaboratively”, and “contrary to those of humanity“ caveats in the paragraph you are citing.
Anyway, something to ponder.)