Where do you see they were training the model? It seemed to me they were evaluating it, which doesn’t cause the same sort of evolutionary pressure.
Yair Halberstadt
Karma: 4,614
There aren’t existing legal remedies. Rather the law is exceedingly unclear and it’s possible that prosecutors will try to throw the book at them, and not at all clear they would succeed.
This proposal makes clear exactly what is and isn’t prosecutable. Prosecutors can decide to, but may choose not to, prosecute in all the above cases.
Note I’m not actually arguing for making AIs a legal person, but the current list of legal persons includes companies, unions, municipalities, ships, rivers and deities.
In fact AIs have one of the most important characteristics often required to make something a legal person—namely they can (and already do) consider the law when deciding whether to do something.
However they lack a suitable definition of long term selfhood, and likely cannot be meaningfully punished, which is the main argument against treating them as legal persons.
Nothing special about chain of thought, I’m happy to use activations, or j space, or the actual response, or just judge based on outcomes and our best intuition. The point is to avoid cases where the AI is clearly “innocent” in the sense that it didn’t know that the person who it was advising on how to buy a gun was a terrorist, and wouldn’t have been expected to know.
Again I’m not interested in holding the AI to account, but the company that deploys it, you seem to be ascribing to me some sort of weird metaphysical obsession with holding AIs to justice rather than offering a practical way of forcing companies to tighten up their game.
But I don’t think it’s higher than increasing parameters to compensate for shorter COT?
You should both be liable, in different ways, as if Claude was an Anthropic employee you were talking to.
So if you ask it to commit a crime, you are liable because that’s illegal. If it commits that crime Anthropic is also liable for the same reason (unless there was no way for Claude to realise it was committing a crime).
If you ask it something innocuous and it commits a crime on the process of fulfilling your request, that’s on Anthropic, for badly training and safeguarding Claude.
An AI has far more in common with a person than a machine given the range of outcomes that can result from a short input by the user.
You do not ask a tractor to plant some seeds and then have it break into the neighbours tool shed and steal some seeds.
We don’t need metaphysics, I am making no statement about AI consciousness whatsoever.
The point is, when the AI hacks into something you look at it’s COT and check if it realised it was hacking into something or not. Similar if it aided a crime.
They almost definitely would prosecute the company if this became a regular pattern (and not just a one off).
Even in civil liability, disclaimers will generally be voided by courts for gross negligence or intentional misconduct.
It could be argued this makes it harder for open source. If a company has a choice between deploying their own instance of Kimi, and taking on any risk themselves, or paying for Claude and letting anthropic take the risk, who are they going to pick?
Hugging face couldn’t do a civil suit because they haven’t been meaningfully harmed.
Federal prosecutors couldn’t do a criminal suit, because there was no intent from open AI, which is required to prosecute cyber security crimes.
Note whoever runs the model still takes on the risk, and it’s difficult to run frontier models on your laptop.
If we are still worried we can legislate to control open source models separately, through some other mechanism.
If a model was asked to research a topic and stole the results from a competitor.
If a model gave concrete advice about how to carry out a terrorist attack.
If a model agreed to take control of a car and crashed it into someone.
If...
I agree these cases are not particularly problematic. This is preparation for worse cases, and also provides a standard which can be used to clarify existing cases so companies can proceed with confidence as to what they need to be worried about and what not.
The reason is that we have existing criminal law and want to apply it to what the AI agent does.
And yes, that proposal of treating an individual as if he was the employer of the LLM is a reasonable approach.
This only applies if you deploy the AI yourself, so a tiny percentage of users (especially since most of the risk is from frontier models).
However I agree that we should be more lenient with them, at the point where we draft legislation we can haggle on the finer points.
I don’t think it would require shutting down the most advanced AIs. If an employee at OpenAI hacked into hugging face, OpenAI might get a fine, but would almost certainly not be shut down. It would incentivise them to invest a bit more in security when training a modified version of their most advanced LLM specifically on cyber security exploits, which I don’t think is a bad thing...
( To be more explicit—my assumption is that AI companies will be occasionally found liable, and rapped on the hands, but only the most irresponsible will end up being forced to shut down over it)
The example above with anthropic I thought made that clear, but I’ll rewrite it.
I think in guidance for judges it should be made clear that the purpose of the legislation is not to shut down the companies, and that amounts imposed should be reasonable and not excessive.