I’m not assuming that the only person with Llama 5 is the one intent on causing harm. Instead, I unfortunately think the sphere of biological attacks is, at least currently, much more favorable to attackers than defenders.
If the biology-as-a-service orgs are following your suggestion to screen for pathogens
I’m not sure we get to assume that? Screening is far from universal today, and not mandatory.
their Llama-5 is going to say, ah, this is exactly what a terrorist would ask for if they were trying to trick us into making a pathogen
This only works if the screener has enough of the genome at once that Llama 5 can figure out what it does, but this is easy to work around.
In general, beyond just bioattack scenarios, any argument purporting to demonstrate dangers of open source LLMs must assume that the defenders also have access
Sure!
If that’s the case, it argues for an approach similar to delayed disclosure policies in computer security: if a new model enables attacks against some existing services, give them early access and time to fix it, then proceed with wide release.
I don’t actually disagree with this! The problem is that the current state of biosecurity is so bad that we need to fix quite a few things first. Once we do have biology as a service KYC, good synthesis screening, restricted access to biological design tools, metagenomic surveillance, much better PPE, etc, then I don’t see Llama 5 as making us appreciably less safe from bioattacks. But that’s much more than 90d! I get deeper into this in Biosecurity Culture, Computer Security Culture.
I’m not assuming that the only person with Llama 5 is the one intent on causing harm. Instead, I unfortunately think the sphere of biological attacks is, at least currently, much more favorable to attackers than defenders.
I’m not sure we get to assume that? Screening is far from universal today, and not mandatory.
This only works if the screener has enough of the genome at once that Llama 5 can figure out what it does, but this is easy to work around.
Sure!
I don’t actually disagree with this! The problem is that the current state of biosecurity is so bad that we need to fix quite a few things first. Once we do have biology as a service KYC, good synthesis screening, restricted access to biological design tools, metagenomic surveillance, much better PPE, etc, then I don’t see Llama 5 as making us appreciably less safe from bioattacks. But that’s much more than 90d! I get deeper into this in Biosecurity Culture, Computer Security Culture.