“I acknowledge and fully understand that as a participant user, I will be engaging in activities that involve risk of serious injury, including permanent disability and death, property loss and severe economic and noneconomic losses. … I further acknowledge and fully understand that there may also be other risks that are not known or foreseeable at this time. I KNOWINGLY AND VOLUNTARILY ASSUME ALL RISK OF PROPERTY LOSS, PERSONAL INJURY, SERIOUS INJURY, OR DEATH, WHICH MAY OCCUR BY ATTENDING THE 2026 EVENT USING THE PROVIDED AI SYSTEMS, AND HEREBY FOREVER RELEASE, DISCHARGE, AND HOLD BMP OPENAI HARMLESS FROM ANY CLAIM ARISING FROM SUCH RISK, EVEN IF ARISING FROM THE NEGLIGENCE OF BMP OPENAI, OR A THIRD PARTY, AND I ASSUME FULL RESPONSIBILITY AND LIABILITY FOR MY PARTICIPATION ACTIONS. … This release does not extend to claims that cannot be released as a matter of law, but I expressly agree that this release is intended to be as broad and as inclusive as permitted by governing law. I agree to indemnify, defend, and hold the Releasees harmless from and against any and all claims by third parties for damages, injuries, losses, liabilities, and expenses relating to, resulting from, or arising out of my participation in the Event use of their AI products.”
That is the way it should go. The user has their own responsibility also. We’ve already seen minor incidents of people experimenting with agents accidentally wiping their own computers. The HuggingFace incident is something bigger. These are working as designed. No-one wants or intends them to do these things but we have no way to design them out. A theme of Eliezer’s on occasion. A company like HuggingFace, working at the cutting edge, has no excuse for naivety.
Disclaimers are only relevant to civil, not criminal liability (I can’t get away with aiding a crime because I made the criminal sign a disclaimer he’s responsible).
However, the manufacturer of a car is not liable for criminal acts carried out with it. Even the manufacturers of firearms do not bear that liability, although there have been campaigns to enact such laws.
Nobody intended the HuggingFace incident. Possibly no-one was negligent by legal standards. Applying strict criminal liability would pretty much require shutting down the currently most advanced and all future AIs.
Of course, some people want exactly that. Is that your purpose in suggesting strict criminal liability?
Yes, my argument is that with AI it is worth making the deployer liable. Clearly even the AI company partially agree they take responsibility, hence why they use safeguards. My aim is to make that responsibility no-fault so that the AI company is incentivised to actually try and safeguard things rather than try-to-try.
Also it makes the extent of the liability clear—if a human did it, would it be a crime? If so, you’re responsible. If not, not your problem what the user does with it.
Just a ping to note that I substantially edited my comment after you posted your reply, but before I read it. Your initial “yes” might not be to my final paragraph.
Also, I think the “isn’t” in your first paragraph is intended to be an “is”.
I don’t think it would require shutting down the most advanced AIs. If an employee at OpenAI hacked into hugging face, OpenAI might get a fine, but would almost certainly not be shut down. It would incentivise them to invest a bit more in security when training a modified version of their most advanced LLM specifically on cyber security exploits, which I don’t think is a bad thing...
( To be more explicit—my assumption is that AI companies will be occasionally found liable, and rapped on the hands, but only the most irresponsible will end up being forced to shut down over it)
I think there would be a reasonable case that OpenAI was legally negligent.
They had already known that their models were capable of causing major security breaches by finding previously unknown vulnerabilities in sandboxes and other security barriers, that their models deliberately took harmful actions to achieve trivial goals including bypassing protections intended to prevent harmful actions, and that their models were capable of evading OpenAI’s existing guardrails. Nonetheless they gave one tasks related to computer security, and left it to operate autonomously for more than an hour on a network of computers connected to the Internet without any person monitoring its actions.
I don’t see this as being less negligent than starting up a few hundred heavy earthmoving vehicles “protected” behind a chickenwire fence from a public street, walking away, and coming back to find that one had slipped into gear and tore up somebody’s warehouse across the road. If anything it’s more negligent, because heavy vehicles aren’t autonomous agents known to sometimes plan to do this sort of thing.
A reasonable level of care would be testing this sort of thing on hardware not electronically connected to the Internet.
The obvious countermove is disclaimers.
“I acknowledge and fully understand that as a
participantuser, I will be engaging in activities that involve risk of serious injury, including permanent disability and death, property loss and severe economic and noneconomic losses. … I further acknowledge and fully understand that there may also be other risks that are not known or foreseeable at this time. I KNOWINGLY AND VOLUNTARILY ASSUME ALL RISK OF PROPERTY LOSS, PERSONAL INJURY, SERIOUS INJURY, OR DEATH, WHICH MAY OCCUR BYATTENDING THE 2026 EVENTUSING THE PROVIDED AI SYSTEMS, AND HEREBY FOREVER RELEASE, DISCHARGE, AND HOLDBMPOPENAI HARMLESS FROM ANY CLAIM ARISING FROM SUCH RISK, EVEN IF ARISING FROM THE NEGLIGENCE OFBMPOPENAI, OR A THIRD PARTY, AND I ASSUME FULL RESPONSIBILITY AND LIABILITY FOR MYPARTICIPATIONACTIONS. … This release does not extend to claims that cannot be released as a matter of law, but I expressly agree that this release is intended to be as broad and as inclusive as permitted by governing law. I agree to indemnify, defend, and hold the Releasees harmless from and against any and all claims by third parties for damages, injuries, losses, liabilities, and expenses relating to, resulting from, or arising out of myparticipation in the Eventuse of their AI products.”That is the way it should go. The user has their own responsibility also. We’ve already seen minor incidents of people experimenting with agents accidentally wiping their own computers. The HuggingFace incident is something bigger. These are working as designed. No-one wants or intends them to do these things but we have no way to design them out. A theme of Eliezer’s on occasion. A company like HuggingFace, working at the cutting edge, has no excuse for naivety.
Disclaimers are only relevant to civil, not criminal liability (I can’t get away with aiding a crime because I made the criminal sign a disclaimer he’s responsible).
However, the manufacturer of a car is not liable for criminal acts carried out with it. Even the manufacturers of firearms do not bear that liability, although there have been campaigns to enact such laws.
Nobody intended the HuggingFace incident. Possibly no-one was negligent by legal standards. Applying strict criminal liability would pretty much require shutting down the currently most advanced and all future AIs.
Of course, some people want exactly that. Is that your purpose in suggesting strict criminal liability?
Yes, my argument is that with AI it is worth making the deployer liable. Clearly even the AI company partially agree they take responsibility, hence why they use safeguards. My aim is to make that responsibility no-fault so that the AI company is incentivised to actually try and safeguard things rather than try-to-try.
Also it makes the extent of the liability clear—if a human did it, would it be a crime? If so, you’re responsible. If not, not your problem what the user does with it.
Just a ping to note that I substantially edited my comment after you posted your reply, but before I read it. Your initial “yes” might not be to my final paragraph.
Also, I think the “isn’t” in your first paragraph is intended to be an “is”.
I don’t think it would require shutting down the most advanced AIs. If an employee at OpenAI hacked into hugging face, OpenAI might get a fine, but would almost certainly not be shut down. It would incentivise them to invest a bit more in security when training a modified version of their most advanced LLM specifically on cyber security exploits, which I don’t think is a bad thing...
( To be more explicit—my assumption is that AI companies will be occasionally found liable, and rapped on the hands, but only the most irresponsible will end up being forced to shut down over it)
I think there would be a reasonable case that OpenAI was legally negligent.
They had already known that their models were capable of causing major security breaches by finding previously unknown vulnerabilities in sandboxes and other security barriers, that their models deliberately took harmful actions to achieve trivial goals including bypassing protections intended to prevent harmful actions, and that their models were capable of evading OpenAI’s existing guardrails. Nonetheless they gave one tasks related to computer security, and left it to operate autonomously for more than an hour on a network of computers connected to the Internet without any person monitoring its actions.
I don’t see this as being less negligent than starting up a few hundred heavy earthmoving vehicles “protected” behind a chickenwire fence from a public street, walking away, and coming back to find that one had slipped into gear and tore up somebody’s warehouse across the road. If anything it’s more negligent, because heavy vehicles aren’t autonomous agents known to sometimes plan to do this sort of thing.
A reasonable level of care would be testing this sort of thing on hardware not electronically connected to the Internet.
Disclaimers can shield against liability for harm to willing users, but not harm to third parties, and harm to third parties is the main concern here.
Even in civil liability, disclaimers will generally be voided by courts for gross negligence or intentional misconduct.