I feel like there are mainly two potential issues with this:
if one thinks AI has huge potential for good, then this would 100% significantly hamper that, because it creates the classic extremely risk averse “cover-your-ass” sort of incentives that have similar effects on many other fields already. This is really a divide about how pessimistic one is about AI outcomes, and thus how much utility is lost by limiting them this way.
I don’t like the “company is at fault for things run on their hardware”. If a user rents hardware from Google to run a model for their own purposes the liability should be with the user, not with Google. Otherwise we incentivise and require levels of surveillance of the companies on their own users that I think instead we should discourage.
If Google runs Kimi on its own hardware, Google is responsible for any actions it takes.
If a private individual runs Deep Seek locally they are responsible for any actions it takes.
In both these cases, the person deploying is also the owner of the hardware.
What kind of cover your ass do you expect companies to take? In some ways this makes it easier for companies—instead of banning all biological research as Claude does now, they only need to ban responding to questions about biological research in a way that would make an individual who responded in the same way criminally culpable. However they need to make extra sure an instance of Claude doesn’t hack into something, even if a user asks.
I feel like there are mainly two potential issues with this:
if one thinks AI has huge potential for good, then this would 100% significantly hamper that, because it creates the classic extremely risk averse “cover-your-ass” sort of incentives that have similar effects on many other fields already. This is really a divide about how pessimistic one is about AI outcomes, and thus how much utility is lost by limiting them this way.
I don’t like the “company is at fault for things run on their hardware”. If a user rents hardware from Google to run a model for their own purposes the liability should be with the user, not with Google. Otherwise we incentivise and require levels of surveillance of the companies on their own users that I think instead we should discourage.
I thought I was clear that the person who deploys it is responsible, not the hardware owner?
Sorry, I was a bit confused by:
In both these cases, the person deploying is also the owner of the hardware.
The example above with anthropic I thought made that clear, but I’ll rewrite it.
Sorry, my bad probably, I’m not very well-rested so I may have not registered that properly.
What kind of cover your ass do you expect companies to take? In some ways this makes it easier for companies—instead of banning all biological research as Claude does now, they only need to ban responding to questions about biological research in a way that would make an individual who responded in the same way criminally culpable. However they need to make extra sure an instance of Claude doesn’t hack into something, even if a user asks.