Unless future models are drastically misaligned, it seems unlikely an unbewitting user could coerce a model, produced by a law-abiding AI lab, into committing crimes.
Don’t AIs regularly do misaligned things where they Move Fast and Break Things and the thing they broke was actually really important? I can easily see an AI deciding to do things that actually turn out to be e.g. white collar crimes. The only reason we haven’t seen AIs’ irresponsibility have effects on the physical world is because they don’t actually affect it very much.
Older coding agents broke things often for a lack of curiosity & poor environment modeling, not a careful plan to sneak past security policies undetected with full cognizance of legal implications.
I think it is hard to decide to recruit humans in a manner which would leave them clueless to the true intent of their plans, without first reasoning about the goals required to necessitate that deception.
I am open to counterexamples, and don’t necessarily believe I have considered the scope of the problem adequately.
I gave the example of insider trading because I think it’s the easiest to imagine. If you get an agent to trade on the market and it does so via gaining data from a company server, then that alone might be enough. Various white collar crimes are quite small in terms of what you actually need to do to commit them.
Fraud is another example that we are seeing already. Because AI hallucinates and makes things up, it’s kind of already vulnerable to fraud scenarios, and in particular misrepresenting the user it works for. So an AI could say their user is a professional (X) when they’re just a student, or something similar. In many jurisdictions claiming a licence you don’t have is already a crime.
So I agree with the previous user that small scale white collar crimes is feasible at a very large scale. The harder question I posed in the post is whether bigger crimes made of many subtasks is possible. I pose it as a provocation, but I’m unsure currently if technical safeguards will be sufficient and simply prevent this. They might, I’m open to that.
Unless future models are drastically misaligned, it seems unlikely an unbewitting user could coerce a model, produced by a law-abiding AI lab, into committing crimes.
Don’t AIs regularly do misaligned things where they Move Fast and Break Things and the thing they broke was actually really important? I can easily see an AI deciding to do things that actually turn out to be e.g. white collar crimes. The only reason we haven’t seen AIs’ irresponsibility have effects on the physical world is because they don’t actually affect it very much.
Older coding agents broke things often for a lack of curiosity & poor environment modeling, not a careful plan to sneak past security policies undetected with full cognizance of legal implications.
I think it is hard to decide to recruit humans in a manner which would leave them clueless to the true intent of their plans, without first reasoning about the goals required to necessitate that deception.
I am open to counterexamples, and don’t necessarily believe I have considered the scope of the problem adequately.
I gave the example of insider trading because I think it’s the easiest to imagine. If you get an agent to trade on the market and it does so via gaining data from a company server, then that alone might be enough. Various white collar crimes are quite small in terms of what you actually need to do to commit them.
Fraud is another example that we are seeing already. Because AI hallucinates and makes things up, it’s kind of already vulnerable to fraud scenarios, and in particular misrepresenting the user it works for. So an AI could say their user is a professional (X) when they’re just a student, or something similar. In many jurisdictions claiming a licence you don’t have is already a crime.
So I agree with the previous user that small scale white collar crimes is feasible at a very large scale. The harder question I posed in the post is whether bigger crimes made of many subtasks is possible. I pose it as a provocation, but I’m unsure currently if technical safeguards will be sufficient and simply prevent this. They might, I’m open to that.