LLMs that significantly help with the creation of bio weapons are 2-3 years away, according to Dario Amodei; hacking capabilities are probably around the same or sooner
So, I note that LLMs only significantly increase the risk of bioterrorist attacks if, indeed, such attacks are currently bottlenecked on knowledge of lab procedures, etc, that LLMs could provide. They could be also bottlenecked on any of the other steps involved—any estimate that LLMs do increase the risk assumes that this is the case.
I am unaware of any paper arguing that such knowledge is indeed the bottleneck.
Note that we also have evidence that such attacks are not currently bottlenecked on such knowledge, but on the (many) other steps involved. He’s a paper from the Future of Humanity Institute that argues that, for instance. So, if the paper is correct, open source LLMs do not contribute to biorisk substantially.
Even apart from the paper, that’s also my prior view—given the relatively weak generalization abilities of LLMs, if they could contribute to knowledge of lab procedures, then the knowledge is already out there and not-too-hard to find.
(The pattern of discourse around biorisk and LLMs looks much more like “ban open source LLMs” was the goal and “use biorisk concerns” was the means, rather than “decrease biorisk” was the goal and “ban open source LLMs was the means.” I’m not saying this about you, to be clear—I’m saying this about the relevant thought-leaders / think tanks who keep mentioning biorisk.)
It now looks like we’re probably going to get medium warning shots on the scale of hundreds of millions of dollars or hundreds of deaths, due to AI-enabled attacks in the next few years. I’m slightly surprised we haven’t seen effective misuse of current open source LLMs, but this seems like mostly a matter of time.
I don’t know what kind of AI-enabled-cyber-attacks causing hundreds of deaths you mean. Right now, if I want to download penetration tools to hack other computers without using any LLM at all I can just do so. What kind of misuse of current open source LLMs, enabling hundreds of deaths, did you expect to have seen?
I appreciate you raising this point about whether LLMs alleviate the key bottlenecks on bioterrorism. I skimmed the paper you linked, thought about my previous evidence, and am happy to say that I’m much less certain that I was before.
My previous thinking for why I believe LLMs exacerbate biosecurity risks:
Kevin Esvelt said so on the 80k podcast, see also the small experiment he mentions. Okay evidence. (I have not listened to the entire episode)
Anthropic says so in this blog post. Upon reflection I think this is worse evidence than I previously thought (seems hard to imagine seeing the opposite conclusion from their research, given how vague the blog post is; access to their internal reports would help).
The Montague 2023 paper you link: the main bottleneck to high consequence biological attacks is actually R&D to create novel threats, especially spread testing which needs to take part in a realistic deployment environment. This requires both being purposeful and being having a bunch of resources, so policies need not be focused on decreasing democratic access to biotech and knowledge.
I don’t find the paper’s discussion of ‘why extensive spread testing is necessary’ to be super convincing, but it’s reasonable and I’m updating somewhat toward this position. That is, I’m not convinced either way. I would have a better idea if I knew how accurate a priori spread forecasting has been for bioweapons programs in the past.
I think the “LLMs exacerbate biosecurity risks” still goes through even if I mostly buy Montague’s arguments, given that those arguments are partially specific to high consequence attacks. Additionally, Montague is mainly arguing that democratization of biotech / info doesn’t help with the main barriers, not that it doesn’t help at all:
The reason spread-testing has not previously been perceived as a the defining stage of difficulty in the biorisk chain (see Figure 1), eclipsing all others, is that, until recently, the difficulties associated with the preceding steps in the risk chain were so high as to deter contemplation of the practical difficulties beyond them. With the advance of synthetic biology enabled by bioinformatic inferences on ’omics data, the perception of these prior barriers at earlier stages of the risk chain has receded.
So I think there exists some reasonable position (which likely doesn’t warrant focusing on LLM --> biosecurity risks, and is a bit of an epistemic cop out): LLMs increase biosecurity risks marginally even though they don’t affect the main blockers for bio weapon development.
Thanks again for your comment, I appreciate you pointing this out. This was one of the most valuable things I’ve read in the last 2 weeks.
I’m not saying LLMs necessarily raise the severity ceiling on either a bio or cyber attack. I think it’s quite possible that AIs will do so in the future, but I’m less worried about this on the 2-3 year timeframe. Instead, the main effect is decreasing the cost of these attacks and enabling more actors to execute such attacks. (as noted, it’s unclear whether this substantially worsens bio threats)
if I want to download penetration tools to hack other computers without using any LLM at all I can just do so
Yes, it’s possible to launch cyber attacks currently. But with AI assistance it will require less personal expertise and be less costly. I am slightly surprised that we have not seen a much greater amount of standard cybercrime (the bar I was thinking when I wrote this was not the hundreds of deaths bar, it was more like “statistically significant increase in cybercrime / serious deepfakes / misinformation, in a way that concretely impacts the world, compared to previous years”).
So, I note that LLMs only significantly increase the risk of bioterrorist attacks if, indeed, such attacks are currently bottlenecked on knowledge of lab procedures, etc, that LLMs could provide. They could be also bottlenecked on any of the other steps involved—any estimate that LLMs do increase the risk assumes that this is the case.
I am unaware of any paper arguing that such knowledge is indeed the bottleneck.
Note that we also have evidence that such attacks are not currently bottlenecked on such knowledge, but on the (many) other steps involved. He’s a paper from the Future of Humanity Institute that argues that, for instance. So, if the paper is correct, open source LLMs do not contribute to biorisk substantially.
Even apart from the paper, that’s also my prior view—given the relatively weak generalization abilities of LLMs, if they could contribute to knowledge of lab procedures, then the knowledge is already out there and not-too-hard to find.
(The pattern of discourse around biorisk and LLMs looks much more like “ban open source LLMs” was the goal and “use biorisk concerns” was the means, rather than “decrease biorisk” was the goal and “ban open source LLMs was the means.” I’m not saying this about you, to be clear—I’m saying this about the relevant thought-leaders / think tanks who keep mentioning biorisk.)
I don’t know what kind of AI-enabled-cyber-attacks causing hundreds of deaths you mean. Right now, if I want to download penetration tools to hack other computers without using any LLM at all I can just do so. What kind of misuse of current open source LLMs, enabling hundreds of deaths, did you expect to have seen?
Thanks for your comment!
I appreciate you raising this point about whether LLMs alleviate the key bottlenecks on bioterrorism. I skimmed the paper you linked, thought about my previous evidence, and am happy to say that I’m much less certain that I was before.
My previous thinking for why I believe LLMs exacerbate biosecurity risks:
Kevin Esvelt said so on the 80k podcast, see also the small experiment he mentions. Okay evidence. (I have not listened to the entire episode)
Anthropic says so in this blog post. Upon reflection I think this is worse evidence than I previously thought (seems hard to imagine seeing the opposite conclusion from their research, given how vague the blog post is; access to their internal reports would help).
The Montague 2023 paper you link: the main bottleneck to high consequence biological attacks is actually R&D to create novel threats, especially spread testing which needs to take part in a realistic deployment environment. This requires both being purposeful and being having a bunch of resources, so policies need not be focused on decreasing democratic access to biotech and knowledge.
I don’t find the paper’s discussion of ‘why extensive spread testing is necessary’ to be super convincing, but it’s reasonable and I’m updating somewhat toward this position. That is, I’m not convinced either way. I would have a better idea if I knew how accurate a priori spread forecasting has been for bioweapons programs in the past.
I think the “LLMs exacerbate biosecurity risks” still goes through even if I mostly buy Montague’s arguments, given that those arguments are partially specific to high consequence attacks. Additionally, Montague is mainly arguing that democratization of biotech / info doesn’t help with the main barriers, not that it doesn’t help at all:
So I think there exists some reasonable position (which likely doesn’t warrant focusing on LLM --> biosecurity risks, and is a bit of an epistemic cop out): LLMs increase biosecurity risks marginally even though they don’t affect the main blockers for bio weapon development.
Thanks again for your comment, I appreciate you pointing this out. This was one of the most valuable things I’ve read in the last 2 weeks.
Second smaller comment:
I’m not saying LLMs necessarily raise the severity ceiling on either a bio or cyber attack. I think it’s quite possible that AIs will do so in the future, but I’m less worried about this on the 2-3 year timeframe. Instead, the main effect is decreasing the cost of these attacks and enabling more actors to execute such attacks. (as noted, it’s unclear whether this substantially worsens bio threats)
Yes, it’s possible to launch cyber attacks currently. But with AI assistance it will require less personal expertise and be less costly. I am slightly surprised that we have not seen a much greater amount of standard cybercrime (the bar I was thinking when I wrote this was not the hundreds of deaths bar, it was more like “statistically significant increase in cybercrime / serious deepfakes / misinformation, in a way that concretely impacts the world, compared to previous years”).