If we have an AI arms race, then the ability for the leading company to keep secrets is highly relevant for the maximum lead that the top company can have over competitors, which in turn determines the amount of resources they can invest into safety applications
Difficulty of selectively releasing safety research from an institution
Concrete scenario: Imagine that you are FHI and some government official approaches you and says
“Hey, we are really interested in this AI Safety thing, and are considering investing multiple millions of dollars into establishing a research center on it. We know that this might produce ideas with concrete capabilities applications, but we are planning to keep the results confidential”.
Should you respond with:
“that seems good, since you can just decide not to release the things if they turn out to have too many capabilities applications?”
or
“no, you will likely be worse than the AI industry at keeping dangerous information secret, so this seems like a bad idea”.
Biosecurity and potential dangerous applications
In biosecurity, you have many applications of potentially dangerous technology, often for the development of better medicine. You often have both industry and government research labs doing research into those technologies. Which ones should you differentially encourage, given that a lot of the risk comes from leaking potentially highly dangerous technologies?
Technology arms race dynamics
If we have an AI arms race, then the ability for the leading company to keep secrets is highly relevant for the maximum lead that the top company can have over competitors, which in turn determines the amount of resources they can invest into safety applications
Difficulty of selectively releasing safety research from an institution
Concrete scenario: Imagine that you are FHI and some government official approaches you and says
Should you respond with:
or
Biosecurity and potential dangerous applications
In biosecurity, you have many applications of potentially dangerous technology, often for the development of better medicine. You often have both industry and government research labs doing research into those technologies. Which ones should you differentially encourage, given that a lot of the risk comes from leaking potentially highly dangerous technologies?