I generally don’t find writeups of standards useful, but this piece was an exception. Below, I’ll try to articulate why:
I think AI governance pieces—especially pieces about standards—often have overly vague language. People say things like “risk management practices” or “third-party audits”, phrases that are generally umbrella terms that lack specificity. These sometimes serve as applause lights (whether the author intended this or not): who could really disagree with the idea of risk management?
I liked that this piece (fairly unapologetically) advocates for specific things that labs should be doing. As an added benefit, the piece causes the reader to realize “oh wow, there’s a lot of stuff that our civilization already does to mitigate this other threat—if we were actually prioritizing AI x-risk as seriously as pandemics, there’s a bunch of stuff we’d be doing differently.”
Naturally, there are some areas that lack detail (e.g., how do biolabs do risk assessments and how would AI labs do risk assessments?), but the reader is at least left with some references that allow them to learn more. I think this moves us to an acceptable level of concreteness, especially for a shallow investigation.
I think this is also the kind of thing that I could actually see myself handing to a policymaker or staffer (in reality, since they have no time, I would probably show them a one-pager or two-pager version of this, with this longer version linked).
I’ll likely send this to junior AI governance folks as a solid example of a shallow investigation that could be helpful. In terms of constructive feedback, I think the piece could’ve had a TLDR section (or table) that directly lists each of the recommendations for AI labs and each of the analogs in biosafety. [I might work on such a table and share it here if I produce it].
I generally don’t find writeups of standards useful, but this piece was an exception. Below, I’ll try to articulate why:
I think AI governance pieces—especially pieces about standards—often have overly vague language. People say things like “risk management practices” or “third-party audits”, phrases that are generally umbrella terms that lack specificity. These sometimes serve as applause lights (whether the author intended this or not): who could really disagree with the idea of risk management?
I liked that this piece (fairly unapologetically) advocates for specific things that labs should be doing. As an added benefit, the piece causes the reader to realize “oh wow, there’s a lot of stuff that our civilization already does to mitigate this other threat—if we were actually prioritizing AI x-risk as seriously as pandemics, there’s a bunch of stuff we’d be doing differently.”
Naturally, there are some areas that lack detail (e.g., how do biolabs do risk assessments and how would AI labs do risk assessments?), but the reader is at least left with some references that allow them to learn more. I think this moves us to an acceptable level of concreteness, especially for a shallow investigation.
I think this is also the kind of thing that I could actually see myself handing to a policymaker or staffer (in reality, since they have no time, I would probably show them a one-pager or two-pager version of this, with this longer version linked).
I’ll likely send this to junior AI governance folks as a solid example of a shallow investigation that could be helpful. In terms of constructive feedback, I think the piece could’ve had a TLDR section (or table) that directly lists each of the recommendations for AI labs and each of the analogs in biosafety. [I might work on such a table and share it here if I produce it].