If there had been common knowledge (which I think there probably would have in the absence of Anthropic’s RSP and associated recruitment/marketing/comms efforts) that achieving nation-state robust cybersecurity was not achievable unless very drastic actions were taken, I do think this would have caused some people to change strategies substantially.
My sense is most efforts which seem to be aimed at nation-state robust cybersecurity for AI (outside labs) have been driven by things like the RAND report and some theory of change downstream of pieces like Situational Awareness which argue that governments may push for TS/SCI classified AI development at some point (for some applications)—not necessarily downstream of Anthropic’s RSP (I’ve never heard anyone mention it directly in, for example, any discussion around why SL5 security is important).
That said (a) I agree with you that this goal is impractial for general AI development and it was foolish of Anthropic to commit to something close to security against state-backed attacks (b) I have heard critique of Jason Clinton’s PoV from parts of the AI cybersecurity community that, having never worked in an Intelligence Community cybersecurity role, he lacks information that would update him on the difficulties (c) I remain confused why some still consider SL5 security for AI model weights a tractable or important goal in the field.
not necessarily downstream of Anthropic’s RSP (I’ve never heard anyone mention it directly in, for example, any discussion around why SL5 security is important).
I don’t know what your background is, but to be clear, I am saying that many people who do not work in computer security have made bad strategic decisions because of this. I don’t think almost anyone working in computer security was super misled here.
Maybe our experiences still diverge with that clarification, but it seemed good to check before I dig deeper.
That’s fair! I am mostly thinking of the AI safety community and the parts of it interested in cybersecurity. I had a lot of discussions with people around the funding ecosystem/government AI safety-interested people/AI policy thinktanks around a year ago about the merits of attempting SL5 and never heard a mention of Anthropic’s RSP specifically, although it seems plausible it was a contributing factor for decisions to pursue that direction.
I was thinking here of talking to various people in EA leadership and various other people working in AI safety research (including some people at labs).
I did go to the AI security forum 2 years ago or so and the vibe I got was that people also made some bad strategic decisions at the time vaguely related to this, but people seemed better calibrated than the people who didn’t have any security interests.
My sense is most efforts which seem to be aimed at nation-state robust cybersecurity for AI (outside labs) have been driven by things like the RAND report and some theory of change downstream of pieces like Situational Awareness which argue that governments may push for TS/SCI classified AI development at some point (for some applications)—not necessarily downstream of Anthropic’s RSP (I’ve never heard anyone mention it directly in, for example, any discussion around why SL5 security is important).
That said (a) I agree with you that this goal is impractial for general AI development and it was foolish of Anthropic to commit to something close to security against state-backed attacks (b) I have heard critique of Jason Clinton’s PoV from parts of the AI cybersecurity community that, having never worked in an Intelligence Community cybersecurity role, he lacks information that would update him on the difficulties (c) I remain confused why some still consider SL5 security for AI model weights a tractable or important goal in the field.
I don’t know what your background is, but to be clear, I am saying that many people who do not work in computer security have made bad strategic decisions because of this. I don’t think almost anyone working in computer security was super misled here.
Maybe our experiences still diverge with that clarification, but it seemed good to check before I dig deeper.
That’s fair! I am mostly thinking of the AI safety community and the parts of it interested in cybersecurity. I had a lot of discussions with people around the funding ecosystem/government AI safety-interested people/AI policy thinktanks around a year ago about the merits of attempting SL5 and never heard a mention of Anthropic’s RSP specifically, although it seems plausible it was a contributing factor for decisions to pursue that direction.
I was thinking here of talking to various people in EA leadership and various other people working in AI safety research (including some people at labs).
I did go to the AI security forum 2 years ago or so and the vibe I got was that people also made some bad strategic decisions at the time vaguely related to this, but people seemed better calibrated than the people who didn’t have any security interests.