Part of what’s going on here is that people distinguish between:
Someone who has principles they disagree with, but who appears to hold to those principles under pressure
Someone who does not appear to have any meaningful principles, or who will not actually hold to them under pressure
Let’s say there’s politician X. And let’s say that I disagree with her on a vast range of policy questions. But one day, she faces a choice: She can oppose some piece of corruption vigorously, but doing so will end her very successful political career. And she makes the choice to go down fighting against the corruption.
In this scenario, I’m going to respect X, even if I disagree with her about almost all normal policy questions.
Anthropic has been very clear that they support classical western democracy, and that they believe that this includes providing AI to the military for a wide variety of uses. This includes spying on other countries, and supporting human-in-the-loop weapon systems. This might not be the position I’d choose, in their shoes.
But apparently they do have hard limits, and they’re willing to forgo significant amounts of revenue and risk official retribution rather than cross those limits. Similarly, they refuse to sell to companies linked to the Chinese military.
This is a positive update for me: I figured that Anthropic would bend more readily under pressure, mostly because most US organizations have chosen to do so in the past.
Similarly, Anthropic’s viewpoint seems to the be that someone is going to build superintelligence, and that while this is insanely risky, it’s better if they’re the ones who choose to gamble with all our lives (as opposed to, say, Sam Altman or a Chinese company). I personally think that racing is a terrible plan, because I expect us to almost certainly lose control of superhuman intelligence in the medium-term.
But the recent events involving the DoD updated me towards thinking that Anthropic is at least sincere about their plan to build the best AGI they can, consistent with winning the race to build AGI. Again, I think this is literally the worst idea in human history, but at least Anthropic appears to pursuing the least terrifying version of this idea sincerely.
I do personally expect Anthropic to further weaken their RSP if such is necessary to “win” the race, even if doing so carries a double-digit risk of actual human extinction. I’m not happy about this at all. But when I think about some of their competitors “winning” that race, I’m even more alarmed.
So, I am not necessarily a fan of their particular principles and goals. But insofar as they actually have some principles that they are currently unwilling to sacrifice, I figure that puts them above the replacement-level AI company.
Let’s say there’s politician X. And let’s say that I disagree with her on a vast range of policy questions. But one day, she faces a choice: She can oppose some piece of corruption vigorously, but doing so will end her very successful political career. And she makes the choice to go down fighting against the corruption.
Fighting against corruption means that you are fighting against something people are actually doing. Anthropic’s public position seems to be that they haven’t stood up for any of their principles that they articulated in the past actually prevents the military from doing things that the military wants to do with their software.
Anthropic had principles that against their software being used for disinformation campaigns, the design or use of weapons, censorship, and malicious cyber operations. If they would enforce any of those principles it would actually prevent the US military from doing things that the US military wants to do.
Only having the red lines against things that Anthropic believes the military doesn’t want to do but not against those that the military wants to do is bending a lot under pressure.
Any normal Secretary of Defense would have said: “Okay, thank you for allowing us to do the things with your software that your stated principles limit. We grant you to have some red lines that don’t involve things we want to do with your software, so that you can present yourself as having principles to the public.”
But Pete Hegseth is not a normal Secretary of Defense. He’s a bully who renamed the office to Secretary of War, who’s not letting a company get away with a face-saving compromise. For him it’s a matter of principle. The fact that there’s a maximum of humiliation that Dario is willing to accept is not the same as him being very principled.
Part of what’s going on here is that people distinguish between:
Someone who has principles they disagree with, but who appears to hold to those principles under pressure
Someone who does not appear to have any meaningful principles, or who will not actually hold to them under pressure
Let’s say there’s politician X. And let’s say that I disagree with her on a vast range of policy questions. But one day, she faces a choice: She can oppose some piece of corruption vigorously, but doing so will end her very successful political career. And she makes the choice to go down fighting against the corruption.
In this scenario, I’m going to respect X, even if I disagree with her about almost all normal policy questions.
Anthropic has been very clear that they support classical western democracy, and that they believe that this includes providing AI to the military for a wide variety of uses. This includes spying on other countries, and supporting human-in-the-loop weapon systems. This might not be the position I’d choose, in their shoes.
But apparently they do have hard limits, and they’re willing to forgo significant amounts of revenue and risk official retribution rather than cross those limits. Similarly, they refuse to sell to companies linked to the Chinese military.
This is a positive update for me: I figured that Anthropic would bend more readily under pressure, mostly because most US organizations have chosen to do so in the past.
Similarly, Anthropic’s viewpoint seems to the be that someone is going to build superintelligence, and that while this is insanely risky, it’s better if they’re the ones who choose to gamble with all our lives (as opposed to, say, Sam Altman or a Chinese company). I personally think that racing is a terrible plan, because I expect us to almost certainly lose control of superhuman intelligence in the medium-term.
But the recent events involving the DoD updated me towards thinking that Anthropic is at least sincere about their plan to build the best AGI they can, consistent with winning the race to build AGI. Again, I think this is literally the worst idea in human history, but at least Anthropic appears to pursuing the least terrifying version of this idea sincerely.
I do personally expect Anthropic to further weaken their RSP if such is necessary to “win” the race, even if doing so carries a double-digit risk of actual human extinction. I’m not happy about this at all. But when I think about some of their competitors “winning” that race, I’m even more alarmed.
So, I am not necessarily a fan of their particular principles and goals. But insofar as they actually have some principles that they are currently unwilling to sacrifice, I figure that puts them above the replacement-level AI company.
Fighting against corruption means that you are fighting against something people are actually doing. Anthropic’s public position seems to be that they haven’t stood up for any of their principles that they articulated in the past actually prevents the military from doing things that the military wants to do with their software.
Anthropic had principles that against their software being used for disinformation campaigns, the design or use of weapons, censorship, and malicious cyber operations. If they would enforce any of those principles it would actually prevent the US military from doing things that the US military wants to do.
Only having the red lines against things that Anthropic believes the military doesn’t want to do but not against those that the military wants to do is bending a lot under pressure.
Any normal Secretary of Defense would have said: “Okay, thank you for allowing us to do the things with your software that your stated principles limit. We grant you to have some red lines that don’t involve things we want to do with your software, so that you can present yourself as having principles to the public.”
But Pete Hegseth is not a normal Secretary of Defense. He’s a bully who renamed the office to Secretary of War, who’s not letting a company get away with a face-saving compromise. For him it’s a matter of principle. The fact that there’s a maximum of humiliation that Dario is willing to accept is not the same as him being very principled.