In your scenario, DARPA would probably not try to take over using AI. It’s not their culture. When they realize their AI achieved DSA, they’d likely voluntarily hand over control to the US government.
Also, DARPA or any other AI project does not operate in a vacuum. Likely, people in government would realize DARPA is on its way to DSA and intervene to take over control before it happens.
We already have a situation where we have democracy with a powerful weapon, namely an atomic bomb. I don’t really see how this will necessarily be different. Oppenheimer also didn’t take over.
As a non-American, I’m worried however that this will decrease power in non-American territories even more (higher permanence and granularity of US sovereignty). Remember that >95% of the world population is not American. All these people would have no say over whatever happens afterwards.
I realized that but I think my counterarguments are true for most organizations who would have a realistic chance of building takeover-level AI. As a case in point, Anthropic launched the Glasswing project trying to fix vulnerabilities rather than saying “great now we can hack into banks for our takeover attempt planned in Q3 2027, let’s not tell anyone about these zero days”.
In your scenario, DARPA would probably not try to take over using AI. It’s not their culture. When they realize their AI achieved DSA, they’d likely voluntarily hand over control to the US government.
Also, DARPA or any other AI project does not operate in a vacuum. Likely, people in government would realize DARPA is on its way to DSA and intervene to take over control before it happens.
We already have a situation where we have democracy with a powerful weapon, namely an atomic bomb. I don’t really see how this will necessarily be different. Oppenheimer also didn’t take over.
As a non-American, I’m worried however that this will decrease power in non-American territories even more (higher permanence and granularity of US sovereignty). Remember that >95% of the world population is not American. All these people would have no say over whatever happens afterwards.
The specific mention of DARPA was arbitrary, it was just a stand-in for “whoever builds ASI first”.
I realized that but I think my counterarguments are true for most organizations who would have a realistic chance of building takeover-level AI. As a case in point, Anthropic launched the Glasswing project trying to fix vulnerabilities rather than saying “great now we can hack into banks for our takeover attempt planned in Q3 2027, let’s not tell anyone about these zero days”.