You talk about multiple different actors. One the one hand, there might be the LessWrong community and on the other hand there’s Anthropic.
When Anthropic makes those decisions, the general public isn’t the key audience. Various people in the security community are a key audience. The government is a key audience. Maybe, companies who want to switch to AI agents and investors are also a key audience.
I hear the point you’re making. But I guess it’s not clear to me who the key audience of the model cards is? The red-teaming demonstrations in them (e.g., blackmailing) are exceptionally valuable for the safety community and as ways to engage the general public. It’s not obvious to me that these are directed at the government, investors or companies who want to switch to AI agents?
From the safety community it would be a very strange thing to say something like “Companies should only tell us about safety problems they are having when they can provide extraordinary proof that they have safety problems.”
That’s not how you handle companies dealing with product safety in any domain. You want companies to share information about safety issues even if the evidence for those safety issues is not overwhelming.
You talk about multiple different actors. One the one hand, there might be the LessWrong community and on the other hand there’s Anthropic.
When Anthropic makes those decisions, the general public isn’t the key audience. Various people in the security community are a key audience. The government is a key audience. Maybe, companies who want to switch to AI agents and investors are also a key audience.
I hear the point you’re making. But I guess it’s not clear to me who the key audience of the model cards is? The red-teaming demonstrations in them (e.g., blackmailing) are exceptionally valuable for the safety community and as ways to engage the general public. It’s not obvious to me that these are directed at the government, investors or companies who want to switch to AI agents?
From the safety community it would be a very strange thing to say something like “Companies should only tell us about safety problems they are having when they can provide extraordinary proof that they have safety problems.”
That’s not how you handle companies dealing with product safety in any domain. You want companies to share information about safety issues even if the evidence for those safety issues is not overwhelming.