no amount of agentic LLM capability could substitute controlled precursors, expensive specialized equipment and real-word lab skills required to do something in the CW space and not kill oneself in the process.
Bullshit. Talk to a Chemical engineer; it’s thousands or tens of thousands of dollars for the equipment, well within the reach of even very small terrorist groups. And building the controlled precursors is exactly the type of task o1 was moderately good at, albeit with some mistakes, and which I couldn’t get more recent models to answer me about; I’d be floored if the models couldn’t guide a novice through this, if jailbroken.
Also, all the useful CW agents which could be made in quantity with very limited tools of synthetic chemistry have already been discovered by the 1980s.
Hence the threat from AGI/ASI in that regard is basically negligible, that’s the consensus of topic experts who researched this stuff
Really? Maybe you can link to something or provide other evidence, since I have heard differently from many of the people with relevant expertise; I might not be so confident on my own, despite my research on this and related topics, but I sure as hell have talked to other experts.
With such a small budget as “tens of thousands of dollars” you will only be able to produce an insignificant amount of CW agents usable for scientific research or, at best, an assassination (but if one has such a budget to kill someone, they will just hire a hitman, which is safer and more reliable). For a mass casualty event terrorists need industrial quantities. In 1992-1993 (before the CW Convention and the UNSC Resolution 1540, when producing CW was actually legal in Japan!), it took Aum Shinrikyo $30M ($70M in 2026) to build and equip the Satyan-7 facility:
Despite the safety features and often state-of-the-art equipment and practices, the operation of the facility was very unsafe – one analyst would later describe the cult as having a “high degree of book learning, but virtually nothing in the way of technical skill.”[26]
When the facility developed leaks, buckets were used to contain spills; several technicians inhaled fumes on repeated occasions, developing ‘symptoms ranging from nosebleeds to convulsions’[1]
Producing everything from scratch will take a lot of time and money, for a non-state violent actor it’s generally easier to just acquire high explosives. And the “not kill oneself in the process” part is important, you seem to have ignored it: unlike producing high explosives or illicit drugs, producing CW agents involves not just very toxic but also very corrosive reagents, such as HF. “Book learning” which LLMs could provide is not very useful in that regard, you need tacit skills learned during lab practice.
Have you actually read the paper or have you just asked an LLM to provide a plausible link to back your belief?[1] It doesn’t refute that: it describes that an ML model was able to reinvent the wheel and rediscover already known CW agents, and also point at some other potentially toxic chemicals which were not researched yet. The fact that they were not researched already strongly indicates that these chemicals either have too large molecular weights, are impossible to synthesize on scale, or both.
The reason why it was possible to exhaust the chemical space with manual search in the 20th century in the first place was that MW limitation: for a toxic chemical to be applicable as CW agent (whether in combat or in a terror attack) it must be either a gas at room temperature, a liquid or at least a volatile solid, otherwise it will only really be able to enter the victim’s organism with drink, food or from something like poisoned clothes, a “sting” with a poisoned umbrella etc. A solid toxin like epibatidine with melting temperature around 60 deg C will not work unless you are assassinating someone, in which case see above.
I have heard differently from many of the people with relevant expertise
Judging by the arguments you presented, either your discussion was very shallow or the “relevant expertise” had nothing to do with actual weapons of mass destruction. As for me, unfortunately I discussed that with topic experts in a space under Chatham House Rules but I will pitch them to write something about it in public, and if they do, I will bring it here.
All the other articles claiming to cover the topic we are discussing which I was able to find are paywalled, not on Sci-Hub and don’t have the details in their abstracts so are pretty useless for our discussion
Bullshit. Talk to a Chemical engineer; it’s thousands or tens of thousands of dollars for the equipment, well within the reach of even very small terrorist groups. And building the controlled precursors is exactly the type of task o1 was moderately good at, albeit with some mistakes, and which I couldn’t get more recent models to answer me about; I’d be floored if the models couldn’t guide a novice through this, if jailbroken.
Look here, a 2022 paper that directly refutes that claim!
Really? Maybe you can link to something or provide other evidence, since I have heard differently from many of the people with relevant expertise; I might not be so confident on my own, despite my research on this and related topics, but I sure as hell have talked to other experts.
With such a small budget as “tens of thousands of dollars” you will only be able to produce an insignificant amount of CW agents usable for scientific research or, at best, an assassination (but if one has such a budget to kill someone, they will just hire a hitman, which is safer and more reliable). For a mass casualty event terrorists need industrial quantities. In 1992-1993 (before the CW Convention and the UNSC Resolution 1540, when producing CW was actually legal in Japan!), it took Aum Shinrikyo $30M ($70M in 2026) to build and equip the Satyan-7 facility:
Producing everything from scratch will take a lot of time and money, for a non-state violent actor it’s generally easier to just acquire high explosives. And the “not kill oneself in the process” part is important, you seem to have ignored it: unlike producing high explosives or illicit drugs, producing CW agents involves not just very toxic but also very corrosive reagents, such as HF. “Book learning” which LLMs could provide is not very useful in that regard, you need tacit skills learned during lab practice.
Have you actually read the paper or have you just asked an LLM to provide a plausible link to back your belief?[1] It doesn’t refute that: it describes that an ML model was able to reinvent the wheel and rediscover already known CW agents, and also point at some other potentially toxic chemicals which were not researched yet. The fact that they were not researched already strongly indicates that these chemicals either have too large molecular weights, are impossible to synthesize on scale, or both.
The reason why it was possible to exhaust the chemical space with manual search in the 20th century in the first place was that MW limitation: for a toxic chemical to be applicable as CW agent (whether in combat or in a terror attack) it must be either a gas at room temperature, a liquid or at least a volatile solid, otherwise it will only really be able to enter the victim’s organism with drink, food or from something like poisoned clothes, a “sting” with a poisoned umbrella etc. A solid toxin like epibatidine with melting temperature around 60 deg C will not work unless you are assassinating someone, in which case see above.
Judging by the arguments you presented, either your discussion was very shallow or the “relevant expertise” had nothing to do with actual weapons of mass destruction. As for me, unfortunately I discussed that with topic experts in a space under Chatham House Rules but I will pitch them to write something about it in public, and if they do, I will bring it here.
All the other articles claiming to cover the topic we are discussing which I was able to find are paywalled, not on Sci-Hub and don’t have the details in their abstracts so are pretty useless for our discussion