That would create an enormous incentive for open weights, or at least for putting model weights in a lot more hands. Especially for the riskiest models. Do you want to do that?
It could be argued this makes it harder for open source. If a company has a choice between deploying their own instance of Kimi, and taking on any risk themselves, or paying for Claude and letting anthropic take the risk, who are they going to pick?
They may not have that choice, because Anthropic would be crazy to take on unlimited risk like that.
So Anthropic can either shut down (and maybe that’s good), or start finding creative ways to monetize letting other people run its models (which means giving them the weights). The only real roadblock to that is that the weights aren’t eligible for any copyright protection and therefore can’t really be licensed, but they might be able to find a way around that.
In the “Anthropic shuts down” fork, the only models left are the open ones.
I think in guidance for judges it should be made clear that the purpose of the legislation is not to shut down the companies, and that amounts imposed should be reasonable and not excessive.
When I log into the Claude web interface and start a conversation, we generally say that I am the one running the model, not Anthropic. It is running on Anthropic’s servers, because I pay a monthly subscription fee for that, but I am the person running the model. Are you saying you want Anthropic to be liable for what I do with Claude?
You should both be liable, in different ways, as if Claude was an Anthropic employee you were talking to.
So if you ask it to commit a crime, you are liable because that’s illegal. If it commits that crime Anthropic is also liable for the same reason (unless there was no way for Claude to realise it was committing a crime).
If you ask it something innocuous and it commits a crime on the process of fulfilling your request, that’s on Anthropic, for badly training and safeguarding Claude.
as if Claude was an Anthropic employee you were talking to.
This is your very weird and unhealthy metaphysics again. Claude isn’t an employee of Anthropic. It is a machine.
> So if you ask it to commit a crime, you are liable because that’s illegal. If it commits that crime Anthropic is also liable for the same reason (unless there was no way for Claude to realise it was committing a crime).
I agree I should be liable there, just as I am liable if I buy a car and run my enemy over with it. But the manufacturer of the car should not be.
And again, think about the implications of your qualifiers. There is NEVER a way for Claude to realize anything, because Claude is a machine, it is not the sort of entity that has the capacity to realize. So even on your rule, unless you can convince the judges and jurors of your very weird metaphysics, Anthropic will never be liable. Your proposal would make it more difficult to hold the labs liable, not less.
> If you ask it something innocuous and it commits a crime on the process of fulfilling your request, that’s on Anthropic, for badly training and safeguarding Claude.
It does seem likely that Anthropic would be liable there, because it seems likely that they were negligent. We don’t need strict laibility for that.
That would create an enormous incentive for open weights, or at least for putting model weights in a lot more hands. Especially for the riskiest models. Do you want to do that?
It could be argued this makes it harder for open source. If a company has a choice between deploying their own instance of Kimi, and taking on any risk themselves, or paying for Claude and letting anthropic take the risk, who are they going to pick?
They may not have that choice, because Anthropic would be crazy to take on unlimited risk like that.
So Anthropic can either shut down (and maybe that’s good), or start finding creative ways to monetize letting other people run its models (which means giving them the weights). The only real roadblock to that is that the weights aren’t eligible for any copyright protection and therefore can’t really be licensed, but they might be able to find a way around that.
In the “Anthropic shuts down” fork, the only models left are the open ones.
I think in guidance for judges it should be made clear that the purpose of the legislation is not to shut down the companies, and that amounts imposed should be reasonable and not excessive.
Note whoever runs the model still takes on the risk, and it’s difficult to run frontier models on your laptop.
If we are still worried we can legislate to control open source models separately, through some other mechanism.
It’s easy to get an account on Modal or whatever, though, and given market pressure it could get even easier.
You said that a pure compute provider like that wasn’t who you meant was the “deployer”.
When I log into the Claude web interface and start a conversation, we generally say that I am the one running the model, not Anthropic. It is running on Anthropic’s servers, because I pay a monthly subscription fee for that, but I am the person running the model. Are you saying you want Anthropic to be liable for what I do with Claude?
You should both be liable, in different ways, as if Claude was an Anthropic employee you were talking to.
So if you ask it to commit a crime, you are liable because that’s illegal. If it commits that crime Anthropic is also liable for the same reason (unless there was no way for Claude to realise it was committing a crime).
If you ask it something innocuous and it commits a crime on the process of fulfilling your request, that’s on Anthropic, for badly training and safeguarding Claude.
This is your very weird and unhealthy metaphysics again. Claude isn’t an employee of Anthropic. It is a machine.
> So if you ask it to commit a crime, you are liable because that’s illegal. If it commits that crime Anthropic is also liable for the same reason (unless there was no way for Claude to realise it was committing a crime).
I agree I should be liable there, just as I am liable if I buy a car and run my enemy over with it. But the manufacturer of the car should not be.
And again, think about the implications of your qualifiers. There is NEVER a way for Claude to realize anything, because Claude is a machine, it is not the sort of entity that has the capacity to realize. So even on your rule, unless you can convince the judges and jurors of your very weird metaphysics, Anthropic will never be liable. Your proposal would make it more difficult to hold the labs liable, not less.
> If you ask it something innocuous and it commits a crime on the process of fulfilling your request, that’s on Anthropic, for badly training and safeguarding Claude.
It does seem likely that Anthropic would be liable there, because it seems likely that they were negligent. We don’t need strict laibility for that.