I’ve just read this post, and it is disturbing what arguments Anthropic made about how the US needs to be ahead of China.
I didn’t really catch up to this news, and I think I know where the anti-Anthropic sentiment is coming from now.
I do think that Anthropic only made those arguments in the context of GPU export controls, and trying to convince the Trump administration to do export controls if nothing else. It’s still very concerning, and could undermine their ability to argue for strong regulation in the future.
That said, I don’t agree with the nuclear weapon explanation.
Suppose Alice and Bob were each building a bomb. Alice’s bomb has a 10% chance of exploding and killing everyone, and a 90% chance of exploding into rainbows and lollipops and curing cancer. Bob’s bomb has a 10% chance of exploding and killing everyone, and a 90% chance of “never being used” and having a bunch of good effects via “game theory.”
I think people with ordinary moral views will not be very angry at Alice, but forgive Bob because “Bob’s bomb was built to not be used.”
I don’t believe the nuclear bomb was truly built to not be used from the point of view of the US gov. I think that was just a lie to manipulate scientists who might otherwise have been unwilling to help.
I don’t think any of the AI builders are anywhere close to “building AI not to be used”. This seems even more clear than with nuclear, since AI has clear beneficial peacetime economically valuable uses.
Regulation does make things worse if you believe the regulation will fail to work as intended for one reason or another. For example, my argument that putting compute limits on training runs (temporarily or permanently) would hasten progress to AGI by focusing research efforts on efficiency and exploring algorithmic improvements.
Hi,
I’ve just read this post, and it is disturbing what arguments Anthropic made about how the US needs to be ahead of China.
I didn’t really catch up to this news, and I think I know where the anti-Anthropic sentiment is coming from now.
I do think that Anthropic only made those arguments in the context of GPU export controls, and trying to convince the Trump administration to do export controls if nothing else. It’s still very concerning, and could undermine their ability to argue for strong regulation in the future.
That said, I don’t agree with the nuclear weapon explanation.
Suppose Alice and Bob were each building a bomb. Alice’s bomb has a 10% chance of exploding and killing everyone, and a 90% chance of exploding into rainbows and lollipops and curing cancer. Bob’s bomb has a 10% chance of exploding and killing everyone, and a 90% chance of “never being used” and having a bunch of good effects via “game theory.”
I think people with ordinary moral views will not be very angry at Alice, but forgive Bob because “Bob’s bomb was built to not be used.”
(Dario’s post did not impact the sentiment of my shortform post.)
I don’t believe the nuclear bomb was truly built to not be used from the point of view of the US gov. I think that was just a lie to manipulate scientists who might otherwise have been unwilling to help.
I don’t think any of the AI builders are anywhere close to “building AI not to be used”. This seems even more clear than with nuclear, since AI has clear beneficial peacetime economically valuable uses.
Regulation does make things worse if you believe the regulation will fail to work as intended for one reason or another. For example, my argument that putting compute limits on training runs (temporarily or permanently) would hasten progress to AGI by focusing research efforts on efficiency and exploring algorithmic improvements.