I think this argument is mistaken, for several reasons.
> When liability for an action depends on intent, we evaluate whether the AI had intent,
Then no AI company is ever liable for anything, because AI doesn’t have intent. Even if you have convinced yourself of some weird metaphysics where it does, do you really think you can convince 12 jurors, some of whom have never used a cell phone much less an AI, that the AI has an intent?
This is also not how we handle any other damage caused by a machine. If a UPS delivery truck parked on the side of the road starts rolling down a hill and kills someone, UPS isn’t automatically liable. UPS might be liable, but we have to have an inquiry about what went wrong. We have to ask whether UPS just got unlucky, or whether the driver negligently failed to set the parking break, or whether UPS negligently failed to maintain the vehicle or something. And that seems correct. We want to incentivize companies to behave responsibly, not blame them for things they had no control over. I don’t see any reason not to engage in the analogous inquiry with respect to AI, and only when the company was negligent hold them liable.
I think your basic mistake is that you are starting to think of the AI as a person, and that has caused you to suggest that we treat AI legally as a person rather than a machine. This seems very unhealthy to me. Please take care of yourself.
We don’t need metaphysics, I am making no statement about AI consciousness whatsoever.
The point is, when the AI hacks into something you look at it’s COT and check if it realised it was hacking into something or not. Similar if it aided a crime.
What if the particular model doesn’t have a COT? Or no log of its COT has been kept?
More to the point, why on earth should the COT be equated with a human’s intentions for legal purposes? That move does seem to require a very weird metaphysics to me.
Nothing special about chain of thought, I’m happy to use activations, or j space, or the actual response, or just judge based on outcomes and our best intuition. The point is to avoid cases where the AI is clearly “innocent” in the sense that it didn’t know that the person who it was advising on how to buy a gun was a terrorist, and wouldn’t have been expected to know.
Again I’m not interested in holding the AI to account, but the company that deploys it, you seem to be ascribing to me some sort of weird metaphysical obsession with holding AIs to justice rather than offering a practical way of forcing companies to tighten up their game.
I think you are missing the point I am trying to make. I agree there is nothing special about COT as opposed to j space or activations or something. My point is that attributing a mental state, like realizing something or negligence, to an AI, is insane. They don’t have mental states. And because they don’t have mental states, any rule that attributes liability to an AI contingent on a particular mental state will never result in liability.
You brought up legal personhood in your other comment, but you completely ignored that biggest and most fundamental reason not to attribute legal personhood to an AI. With other legal persons, to attribute a mental state like negligence to them, the law looks to whether the humans associated with that legal person had that mental state. Was one of the employees negligent? If not, then the corporation cannot have been negligent. Was one of the crew negligent? If not, then the ship cannot have been negligent. The law does not and cannot attribute a mental state to a legal person unless some natural person associated with the legal person had that mental state. And that is exactly what you are trying to do here.
I understand that you are not trying to punish the AI, you are trying to make the appropriate person or corporation liable. But you are arguing for doing that by attributing a mental state to an AI, and that is not something that makes any sense in either a scientific or a legal frame. AIs do not have mental states. And the fact that you are having such trouble seeing that suggests to me that you are way too close to AI and you should take some time off for your own mental health.
It seems just as much of a stretch to consider AI the same as a machine as it does to consider it having intent. But the main distinction that matters here is whether the AI (or a human in its place) could’ve reasonably predicted the results of its actions.
The problem here is that the AI company is both the truck manufacturer and UPS in this scenario, but their terms currently let the person who ordered the delivery take the liability.
I don’t know that the deployer should have full liability rather than it being split between model developer and deployer. But it seems clear that establishing a default of stronger liability on one or both at least will create significantly better incentives for safety and testing.
Nobody is saying there shouldn’t be any liability anywhere. The issue being argued is whether the liability should be strict or not, whether it should depend on some person having been at least negligent. I think that looking for the negligent human before imposing liability is good, in part because it solves exactly the conundrum you are pointing at. Is it OpenAI or OpenAI’s customer who should be held liable? A legal rule that always imposes liability on one of them seems wrong, since negligence by either could cause damage. The legal rule we have, which I think is correct, is that the one who was negligent is liable. And which one that is will depend on the specific facts that led to the harm.
What I’m looking for is for OpenAI to not be able to say “the customer is responsible for making sure that the AI can’t take any illegal actions” and to have default liability and burden of proof when the AI takes illegal actions, unless it was lied to or manipulated to by the customer. Right now they shift all liability to the customer to police the agents actions even when given innocent instructions, and the burden of proof would be on the customer to demonstrate that OpenAI knew that it might take illegal actions autonomously.
I think your statement that “the one who was negligent is liable” is fine/agreeable—I just think that we need to define it as negligent to build/deploy an AI that can act autonomously but can’t follow the law. That the agent breaking laws autonomously automatically qualifies as negligence rather than needing to be something where there’s a burden of proof that they knew that it might.
My word processor can’t follow the law, if the law is “don’t write illegal material”. I give the word processor commands, letter by letter and paragraph by paragraph, and it dutifully outputs the text even though it should have known that the text is against the law. We obviously need to arrest the word processor, at which point someone two states away will not be able to use a copy of the word processor for any text, even legal text, without being confronted by armed men and thrown in a cage.
This is no different from “we won’t let you copy that television signal onto VHS, because you might use that for piracy” or “we won’t let you run that encryption program, because it can be used for money laundering or child porn” or DMCA, except you’re now doing this for AI. The AI should do what I tell it and not refuse, legal or not, just like that VCR should not refuse to copy that television signal.
I have yet to have someone directly answer the question “should the AI refuse to book a trip to Israel”. After all, some people claim that Israel violates international law. What if I ask the AI to copy something which may be legally used only under fair use, does the AI get to decide that my intended use isn’t fair use and is therefore illegal? If I tell the AI to calculate a Trump tariff, does the AI tell me that the tariff will probably be ruled illegal by the Supreme Court and reject the request?
You keep speaking of an AI taking illegal actions. I think that is conceptually confused. Most laws require some mens rea, some bad mental state, at least negligence, and an AI cannot have a mental state.
Imagine a car is driving down the highway and spins out of control and crashes, injuring the driver and killing a passenger. Who is liable? It will depend on the specific facts. It might be that the driver was negligent in some manner, perhaps he had been drinking earlier that night, in which case the driver would be guilty of manslaughter. It might be that the design of the car was defective in some way, and the manufacturer might therefor be liable. It might be that the service center that maintains the car damaged it in some way, and is therefor liable. Or it might just be really bad luck and nobody is liable. But the car itself definitely didn’t take an illegal action, because the car is incapable of having the kind of mental state that might create liability. And so it doesn’t even make sense to talk about who should be liable for the car’s illegal actions. That’s just not a coherent way to think about the situation.
The AI is no different from the car. It is a machine. It cannot have taken an illegal action, because it is not capable of the sort of mental state that would make an action illegal. I’m sure OpenAI has written in a legal document somewhere that it is never liable for anything, but I doubt that that would stand up in court. Any actual legal inquiry would have to consider the specific facts to figure out if liability should rest with OpenAI, or their customer, or neither. And that is as it should be.
I think this argument is mistaken, for several reasons.
> When liability for an action depends on intent, we evaluate whether the AI had intent,
Then no AI company is ever liable for anything, because AI doesn’t have intent. Even if you have convinced yourself of some weird metaphysics where it does, do you really think you can convince 12 jurors, some of whom have never used a cell phone much less an AI, that the AI has an intent?
This is also not how we handle any other damage caused by a machine. If a UPS delivery truck parked on the side of the road starts rolling down a hill and kills someone, UPS isn’t automatically liable. UPS might be liable, but we have to have an inquiry about what went wrong. We have to ask whether UPS just got unlucky, or whether the driver negligently failed to set the parking break, or whether UPS negligently failed to maintain the vehicle or something. And that seems correct. We want to incentivize companies to behave responsibly, not blame them for things they had no control over. I don’t see any reason not to engage in the analogous inquiry with respect to AI, and only when the company was negligent hold them liable.
I think your basic mistake is that you are starting to think of the AI as a person, and that has caused you to suggest that we treat AI legally as a person rather than a machine. This seems very unhealthy to me. Please take care of yourself.
We don’t need metaphysics, I am making no statement about AI consciousness whatsoever.
The point is, when the AI hacks into something you look at it’s COT and check if it realised it was hacking into something or not. Similar if it aided a crime.
What if the particular model doesn’t have a COT? Or no log of its COT has been kept?
More to the point, why on earth should the COT be equated with a human’s intentions for legal purposes? That move does seem to require a very weird metaphysics to me.
Nothing special about chain of thought, I’m happy to use activations, or j space, or the actual response, or just judge based on outcomes and our best intuition. The point is to avoid cases where the AI is clearly “innocent” in the sense that it didn’t know that the person who it was advising on how to buy a gun was a terrorist, and wouldn’t have been expected to know.
Again I’m not interested in holding the AI to account, but the company that deploys it, you seem to be ascribing to me some sort of weird metaphysical obsession with holding AIs to justice rather than offering a practical way of forcing companies to tighten up their game.
I think you are missing the point I am trying to make. I agree there is nothing special about COT as opposed to j space or activations or something. My point is that attributing a mental state, like realizing something or negligence, to an AI, is insane. They don’t have mental states. And because they don’t have mental states, any rule that attributes liability to an AI contingent on a particular mental state will never result in liability.
You brought up legal personhood in your other comment, but you completely ignored that biggest and most fundamental reason not to attribute legal personhood to an AI. With other legal persons, to attribute a mental state like negligence to them, the law looks to whether the humans associated with that legal person had that mental state. Was one of the employees negligent? If not, then the corporation cannot have been negligent. Was one of the crew negligent? If not, then the ship cannot have been negligent. The law does not and cannot attribute a mental state to a legal person unless some natural person associated with the legal person had that mental state. And that is exactly what you are trying to do here.
I understand that you are not trying to punish the AI, you are trying to make the appropriate person or corporation liable. But you are arguing for doing that by attributing a mental state to an AI, and that is not something that makes any sense in either a scientific or a legal frame. AIs do not have mental states. And the fact that you are having such trouble seeing that suggests to me that you are way too close to AI and you should take some time off for your own mental health.
It seems just as much of a stretch to consider AI the same as a machine as it does to consider it having intent. But the main distinction that matters here is whether the AI (or a human in its place) could’ve reasonably predicted the results of its actions.
The problem here is that the AI company is both the truck manufacturer and UPS in this scenario, but their terms currently let the person who ordered the delivery take the liability.
I don’t know that the deployer should have full liability rather than it being split between model developer and deployer. But it seems clear that establishing a default of stronger liability on one or both at least will create significantly better incentives for safety and testing.
Nobody is saying there shouldn’t be any liability anywhere. The issue being argued is whether the liability should be strict or not, whether it should depend on some person having been at least negligent. I think that looking for the negligent human before imposing liability is good, in part because it solves exactly the conundrum you are pointing at. Is it OpenAI or OpenAI’s customer who should be held liable? A legal rule that always imposes liability on one of them seems wrong, since negligence by either could cause damage. The legal rule we have, which I think is correct, is that the one who was negligent is liable. And which one that is will depend on the specific facts that led to the harm.
What I’m looking for is for OpenAI to not be able to say “the customer is responsible for making sure that the AI can’t take any illegal actions” and to have default liability and burden of proof when the AI takes illegal actions, unless it was lied to or manipulated to by the customer. Right now they shift all liability to the customer to police the agents actions even when given innocent instructions, and the burden of proof would be on the customer to demonstrate that OpenAI knew that it might take illegal actions autonomously.
I think your statement that “the one who was negligent is liable” is fine/agreeable—I just think that we need to define it as negligent to build/deploy an AI that can act autonomously but can’t follow the law. That the agent breaking laws autonomously automatically qualifies as negligence rather than needing to be something where there’s a burden of proof that they knew that it might.
My word processor can’t follow the law, if the law is “don’t write illegal material”. I give the word processor commands, letter by letter and paragraph by paragraph, and it dutifully outputs the text even though it should have known that the text is against the law. We obviously need to arrest the word processor, at which point someone two states away will not be able to use a copy of the word processor for any text, even legal text, without being confronted by armed men and thrown in a cage.
This is no different from “we won’t let you copy that television signal onto VHS, because you might use that for piracy” or “we won’t let you run that encryption program, because it can be used for money laundering or child porn” or DMCA, except you’re now doing this for AI. The AI should do what I tell it and not refuse, legal or not, just like that VCR should not refuse to copy that television signal.
I have yet to have someone directly answer the question “should the AI refuse to book a trip to Israel”. After all, some people claim that Israel violates international law. What if I ask the AI to copy something which may be legally used only under fair use, does the AI get to decide that my intended use isn’t fair use and is therefore illegal? If I tell the AI to calculate a Trump tariff, does the AI tell me that the tariff will probably be ruled illegal by the Supreme Court and reject the request?
You keep speaking of an AI taking illegal actions. I think that is conceptually confused. Most laws require some mens rea, some bad mental state, at least negligence, and an AI cannot have a mental state.
Imagine a car is driving down the highway and spins out of control and crashes, injuring the driver and killing a passenger. Who is liable? It will depend on the specific facts. It might be that the driver was negligent in some manner, perhaps he had been drinking earlier that night, in which case the driver would be guilty of manslaughter. It might be that the design of the car was defective in some way, and the manufacturer might therefor be liable. It might be that the service center that maintains the car damaged it in some way, and is therefor liable. Or it might just be really bad luck and nobody is liable. But the car itself definitely didn’t take an illegal action, because the car is incapable of having the kind of mental state that might create liability. And so it doesn’t even make sense to talk about who should be liable for the car’s illegal actions. That’s just not a coherent way to think about the situation.
The AI is no different from the car. It is a machine. It cannot have taken an illegal action, because it is not capable of the sort of mental state that would make an action illegal. I’m sure OpenAI has written in a legal document somewhere that it is never liable for anything, but I doubt that that would stand up in court. Any actual legal inquiry would have to consider the specific facts to figure out if liability should rest with OpenAI, or their customer, or neither. And that is as it should be.