Are you open to writing more about this? This is among top 3 most popular arguments against open source AI on lesswrong and elsewhere.
I agree with you you need a group of > 1000 people to manufacture one of those large machines that does phosphoramidite DNA synthesis. The attack vector I more commonly see being suggested is that a powerful actor can bribe people in the existing labs to manufacture a bioweapon while ensuring most of them and most of rest of society remains unaware this is happening.
Many people wrongly assume that the main way to use bioweapons is to create small amount of pathogen to release it in environment with outbreak as an intended outcome. (I assume that where your sentence about DNA synthesis comes from.) The problem is that creating outbreaks in practice is very hard, we, thankfully, don’t know reliable way to do that. In practice, the way that bioweapons work reliably is “bomb-saturate the entire area with anthrax such that first wave of death is going to be from anaphylactic shock rather than infection” and to create necessary amount of pathogen you need industrial infrastructure which doesn’t exist, because nobody in our civilization cultivates anthrax at industrial scale.
Are you open to writing more about this? This is among top 3 most popular arguments against open source AI on lesswrong and elsewhere.
I agree with you you need a group of > 1000 people to manufacture one of those large machines that does phosphoramidite DNA synthesis. The attack vector I more commonly see being suggested is that a powerful actor can bribe people in the existing labs to manufacture a bioweapon while ensuring most of them and most of rest of society remains unaware this is happening.
I’m trying to write post, but, well, it’s hard.
Many people wrongly assume that the main way to use bioweapons is to create small amount of pathogen to release it in environment with outbreak as an intended outcome. (I assume that where your sentence about DNA synthesis comes from.) The problem is that creating outbreaks in practice is very hard, we, thankfully, don’t know reliable way to do that. In practice, the way that bioweapons work reliably is “bomb-saturate the entire area with anthrax such that first wave of death is going to be from anaphylactic shock rather than infection” and to create necessary amount of pathogen you need industrial infrastructure which doesn’t exist, because nobody in our civilization cultivates anthrax at industrial scale.