This could be appropriately dealt with through compute governance to make sure that there aren’t relevant amounts of AI chips in potential blacksites.
Or, if what you’re saying is that countries will just refuse access to inspectors at the sites that are visible, then you can start reaching into the coercive toolbox to punish that behavior and slow down the classified projects regardless.
My understanding is that this is arguing that current LLMs are too slow and expensive for them to be able to run amok online, since they have to be big enough to have the capability to break out of their container, make copies, and raise money for/steal compute. I think this is a bad bet against future models getting much more efficient and cheap to run while still having these capabilities, as well as the AIs just designing lots of very effective self-replicating malware to help progress their goals.
I also don’t doubt we’ll also see AIs capable of self-replicating and maintaining themselves across the internet well before they’re capable of RSI. What’s going to happen to communications infrastructure when its flooded by a swarm of agents chasing random goals and breaking into things to accomplish them?
None of it. I don’t think I could get claude to write up a drone attack on a city even if I wanted to.
To be fair, AI Frontiers has a tone/style standard, so the voice I use in the piece is less indiosyncratic and more editorialized than how I normally write.
They would fail for the points Yair brought up. The original draft included a section on them, but Frontiers articles have soft length limits, so I cut it for its similarity to the kinetic interceptor point.
Basically: Kinetic drones can work, but you either need many more of them or more capable ones to reliably intercept the attacker’s units. Predicting and intercepting a target with the same movement capabilities as you is inherently inefficient, which is the same issue you get with missile interceptors. This is compounded by coverage problems, because the attacker can pour all of their resources into one spot to overwhelm the pre-positioned stock of defenses.
Commodified just means mass produced and standardized. Bombs are heavily commodified because there are extensive supply chains that drive down their costs, even if you can’t easily buy them online. The point is that autonomous drones won’t be handcrafted, bespoke goods that require specialized industry and tooling to produce, but instead easily assembled from cheap components. Once a few major militaries start doing this, the supply of autonomous drones will be enormous, leaving plenty of room for smuggling or proxy transfers to non-state actors.
This is already exactly what happened with standard fixed wing drones. It was expensive to develop the initial concept, and then the costs collapsed to the point that the Houthis are throwing Shaheds around in Yemen.
I’m a bit confused by what you mean about being vulnerable to nuclear weapons: if anything, a terrorist group built of nomadic, mobile cells seems like the ideal candidate for abusing a WMD and absorbing retaliation.
I agree that terrorists won’t be the first to aquire this technology. But once it’s commodified, them purchasing large quantities of drones the same way they smuggle guns and munitions seems very feasible.
I think the primary use of true WMD scale devices has historically been entirely defensive. Chemical weapons are largely not used due to various reasons including complexity and ineffectiveness. Nuclear weapons weapons were used twice, and caused the two most powerful military systems in history, led by relatively bellicose leadership at times, to go to great lengths to avoid anything that could escalate.
I would credit the lack of WMD use to a) the inflexibility/limitations of non-nuclear WMDs, and b) the good fortune that nuclear weapons were bottlenecked on hard industrial and material inputs. If you expanded the number of people with access to nuclear weapons, I don’t think you would see everyone rationally withold from using them forever. I also think this somewhat mischaracterizes the attitude towards nuclear weapons use during the Cold War. Early on, launch authority was often delegated to commanders in the field, and they were very nearly deployed during the tension around the Cuban Missile Crisis and during the Korean War.
Nuclear weapons are also asymmetrically useful for weak states, because they negate conventional military superiority (see: North Korea). They don’t let you take or hold territory, and so don’t assist conquest. But we still would be incredibly worried to see them proliferate widely, as drone swarms are likely to do. Even if they never got cheap enough to quite make their way into the hands of non-state actors, we would be always running the risk that small regional conflicts explode into extremely lethal wars.
The advantage of drones in this regard is that they let you threaten a much larger number of people with a small payload. Short of a nuke, there’s nothing you could put into a shipping container that would come close to threatening a meaningful fraction of a major city. This gives them a deterrent value that never existed for states with lots of ordinary munitions, and so could be worth the cost of acquisition.
For terrorists, acquiring and using such weapons might be more prohibitive than ramming into a crowd with a truck. But the fact that it would be possible at all would encourage people to exploit the vulnerability, especially since drone swarms carry considerable shock and awe value for terror (unlike destroying a water treatment facility).
For the record, I didn’t downvote this comment. I have no gripes with skepticism.
I think this piece was valuable for a few reasons.
First, the national security establishment is generally very conservative about forecasting future technology. Using the example of drone WMDs helps illustrate that existing offensive technology could be ready for WMD use very quickly, and that it’s worth giving the same treatment to other, more speculative technology (i.e., AI enabled weapons).
Second, drones have unique advantages as WMDs. They can be assembled from components that are difficult to sanction, have deniable application for civilian or conventional use, and allow for the kind of precise force projection that’s otherwise prevented chemical and biological weapons from acting as effective non-nuclear deterrents.
Third, people can intellectually realize something is important without following through on the implications of what that would mean. There are many people in natsec that think cheap drones are possible and useful, but many fewer that have sat down to figure out what exactly that will imply over the next couple of years. Pointing out that there are easy ways to siege cities and giving a picture of what it would look like is important to push past compartmentalization.
It’s when you want to kill/disable ONLY combatants or insurgents, and they’re mixed in with civilians you want to protect that you’re restricted from using these very effective indiscriminate weapons. Which is almost always the case.
Aside from the use cases of terrorism and deterrence sadly. I agree that discrimination is useful in most conventional contexts (hence why countries that otherwise could haven’t mass produced these kinds of drones yet), but it’s irrelevant in situations where you are just trying to threaten large numbers of enemy civilians. Unfortunately, I think these drones will inevitably become cheap enough to make sieging cities practical and affordable.
In the limit, I also expect it to become possible to design drones that are smart enough to distinguish targets, but it’s always going to be cheaper to throw on simpler sensors and less compute and ignore this constraint. There’s a terrible incentive to race to the bottom on drone design if this “targeting tax” is large enough.
Plan A for anyone looking for a link. Awesome to finally see this out from the AIFP team!
I used a similar idea for some strike modeling that I posted earlier today (using the AIFP software efficiency framework), and when I measured the impact of shrinking present and future compute of the leading lab by 90% (along with an attack on the AI supply chain) it resulted in a delay to ASI of ~7.5 years for the U.S. and China, trending down to six months as you get closer to striking when ASI would have finished training.
You might say that it was genuinely load-bearing.
This is awesome, but I do have some questions about the potential for future (explosive) superweapons. It seems to me that there is clear potential for an antimatter bomb: if there is some way to efficiently produce antimatter and contain it (in mangetic containment?) then the resulting annihilation with ordinary matter should be orders of magnitude more destructive per gram of “explosive” than a thermonuclear warhead. Many speculative space colonization frameworks often include descriptions of antimatter as an extremely mass-efficient fuel for acceleration and deceleration of Von Neuman probes, but I haven’t yet seen any arguments that point out that this technology is also a dual-use global WMD.
Are there fundamental physical constraints that make it impractical to either produce or contain antimatter for this purpose?
Thank you both!
I see e/acc as less a response to AI x-risk than as a general symptom of right-wing Vicemaxxing, where frustration with moralizing woke politics made it fashionable to signal that you had no morals at all. Rather than have a bunch of people who had a principled disagreement with the AI safety arguments and so allied with their right-wing political rivals, they mostly started out as political rivals who instinctively inverted the positions of the blue-tribe AI discourse once they encountered them for the first time.
I personally think more AIxNuclear stuff would be very valuable, especially if it’s from a properly ASI-pilled perspective. A lot of existing writing on AI‘s effect on nuclear deterrence is either mired in an AI as a Normal Technology perspective, or doesn’t follow the implications all the way through to the end. There’s plenty of natsec people that will agree that AGI is a Big Deal (TM), but refuse to consider what happens if that lets you do crazy things like automating your industrial base and get it to grow exponentially.
There’s two ideas mixed together here in this comment, some of which I’m planning on covering in another piece on the effects of automation on deterrence.
The first is something I’ve called the “size” of a decisive strategic advantage. Agents that primarily care about future outcomes/are extremely reconstitutable/highly risk tolerant need a much smaller technological or military lead in order to be willing to strike and secure a DSA, since they are more willing to absorb immediate damage for long-term strategic gain. Agents that value existing assets (e.g. states caring about their civilians), on the other hand, are much easier to threaten, and so need to be so strategically superior that they are not only guaranteed to win a conflict but to do so in an essentially costless way. A democratic U.S. would not try to overpower the Chinese government with new AI-enabled superweapons unless it was basically certain it could do so without suffering a catastrophic response.
This applies to AIs or states! An authoritarian country would have fewer qualms about the sacrifices they need to make for long-term security than a democratic one, nevermind a misaligned AI. It’s just a question of how easy the things the agent values are to threaten.
The second, related idea is that this creates a perverse strategic incentive to stop caring about your civilians once you’ve automated away the instrumental need to defend them. This is the military version of the gradual disempowerment story: once the industrial base is automated, and if the key decision makers are secured from consequences, it becomes easier to trade off on the security of your civilians for strategic gain. I think the primary risk here is less leaders jumping the gun to “win” a nuclear exchange (or whatever the equivalent is for future superweapons), and moreso that it will make it more attractive to do dangerous things like capture resources or negotiate under brinksmanship. Not that it’s any better to die in an accidental war over a U.S. demand for a bigger slice of lunar surface area than a direct war of conquest in mainland China, of course.
Compute governance is only necessary to the extent that you can confidently know where large amounts of compute are, which I would describe as a relatively easy problem given how hard it is to hide fabs and how simple choking off production could be. In order to sabotage their operations, you don’t need to actually know what they’re using them for: the very fact that they’re not revealing it is a justification for sabotage.
This same basic setup is already how nuclear enrichment sites are handled. Nuclear enrichment facilities are very hard to hide outright, so states can be very confident about whether a rival is able to pursue a nuclear weapons program. If enrichment sites are observed, and if IAEA inspectors are refused access, then states start applying coercive leverage up to and including outright destroying the facilities.