Judging by your writing I think you missed a new paper red-teaming DNA synthesis screening software. They’re using AI to create proteins that function the same but with different amino acids (probably conformation-based) which bypasses the DNA screening, because screening (probably) isn’t translating and throwing it into Alphafold and comparing it to known toxins. Though the paper didn’t test whether the AI-generated toxins are actually toxic, but we can assign that a reasonable probability.
I hadn’t! Thanks for pointing this out! Looks like someone is actually on top of this after all.
That being said...translating DNA to protein with standard codon table is just one encoding scheme. And we can recode organisms to use a different encoding scheme. And no DNA screening would be able to catch it, since they have no knowledge of the nonstandard codon table you’re using
Who do you mean by “we” in this case? I had the vague vibes that the “we” in question is “Jason Chin’s lab and a few other similarly high-level groups elsewhere”, but that it was pretty far beyond the capabilities of the marginal bioterrorist, and anyone who’s successfully fucking around with nonstandard codon sequences can definitely already make a bioweapon. Is this incorrect under your model?
I’m not sure a marginal bioterrorist can train an AI model to obfuscate a DNA sequence to bypass the sequence scanning, but I concede it’s definitely easier to do than recode an organism. I’m not really sure what the skillset/resources of a marginal bioterrorist are.
But we should watch out for proliferation/commercialization of recoded organisms, since recoded organisms would be easier to recode further (if they introduce a new codon, just modify the synthetase to load a different amnio acid).
A marginal bioterrorist could probably just brew up a vat of anthrax which technically counts. Advanced labs definitely have more capacity for modification, but they still need to source the pathogens.
Yes—they made a huge number of mistakes, despite having sophisticated people and tons of funding. It’s been used over and over to make the claim that bioweapons are really hard—but I do wonder how much using an LLM for help would avoid all of these classes of mistake. (How much prosaic utility is there for project planning in general? Some, but at high risk if you need to worry about detection, and it’s unclear that most people are willing to offload or double check their planning, despite the advantages.)
I hadn’t! Thanks for pointing this out! Looks like someone is actually on top of this after all.
Who do you mean by “we” in this case? I had the vague vibes that the “we” in question is “Jason Chin’s lab and a few other similarly high-level groups elsewhere”, but that it was pretty far beyond the capabilities of the marginal bioterrorist, and anyone who’s successfully fucking around with nonstandard codon sequences can definitely already make a bioweapon. Is this incorrect under your model?
I’m not sure a marginal bioterrorist can train an AI model to obfuscate a DNA sequence to bypass the sequence scanning, but I concede it’s definitely easier to do than recode an organism. I’m not really sure what the skillset/resources of a marginal bioterrorist are.
But we should watch out for proliferation/commercialization of recoded organisms, since recoded organisms would be easier to recode further (if they introduce a new codon, just modify the synthetase to load a different amnio acid).
A marginal bioterrorist could probably just brew up a vat of anthrax which technically counts. Advanced labs definitely have more capacity for modification, but they still need to source the pathogens.
Perhaps worth nothing that they’ve tried in the past, and failed.
It seems they were literally using the nonpathogenic attenuated anthrax strain used for vaccines.
Yes—they made a huge number of mistakes, despite having sophisticated people and tons of funding. It’s been used over and over to make the claim that bioweapons are really hard—but I do wonder how much using an LLM for help would avoid all of these classes of mistake. (How much prosaic utility is there for project planning in general? Some, but at high risk if you need to worry about detection, and it’s unclear that most people are willing to offload or double check their planning, despite the advantages.)