I hear the point you’re making. But I guess it’s not clear to me who the key audience of the model cards is? The red-teaming demonstrations in them (e.g., blackmailing) are exceptionally valuable for the safety community and as ways to engage the general public. It’s not obvious to me that these are directed at the government, investors or companies who want to switch to AI agents?
From the safety community it would be a very strange thing to say something like “Companies should only tell us about safety problems they are having when they can provide extraordinary proof that they have safety problems.”
That’s not how you handle companies dealing with product safety in any domain. You want companies to share information about safety issues even if the evidence for those safety issues is not overwhelming.
I hear the point you’re making. But I guess it’s not clear to me who the key audience of the model cards is? The red-teaming demonstrations in them (e.g., blackmailing) are exceptionally valuable for the safety community and as ways to engage the general public. It’s not obvious to me that these are directed at the government, investors or companies who want to switch to AI agents?
From the safety community it would be a very strange thing to say something like “Companies should only tell us about safety problems they are having when they can provide extraordinary proof that they have safety problems.”
That’s not how you handle companies dealing with product safety in any domain. You want companies to share information about safety issues even if the evidence for those safety issues is not overwhelming.