Fair point, valley9. I don’t think a little bit of agency throws you into an entirely different regime. It’s more that I think that the more powerful an agent you build, the more it is able to autonomously change the world to work with goals, the more you move into dangerous territory. But also, it’s going to tempt people. Somebody out there is going to be tempted to say, “go make me money, just don’t get caught doing anything illegal in a way that gets traced back to me.”
That command given to a sufficiently powerful AI system could have a lot of dangerous results.
But also, it’s going to tempt people. Somebody out there is going to be tempted to say, “go make me money, just don’t get caught doing anything illegal in a way that gets traced back to me.” That command given to a sufficiently powerful AI system could have a lot of dangerous results.
Indeed. This seems like more of a social problem than an alignment problem though: ensure that powerful AIs tend to be corporate AIs with corporate liability rather than open-source AIs, and get the AIs to law enforcement (or even law enforcement “red teams”—should we make that a thing?) before they get to criminals. I don’t think improving aimability helps guard against misuse.
I don’t think improving aimability helps guard against misuse.
I think needs to be stated more clearly: Alignment and Misuse are very different things, so much so that what policies and research work for one problem will often not work on another problem, and the worlds of misuse and misalignment are quite different.
Though note that the solutions for misuse focused worlds and structural risk focused worlds can work against each other.
Also, this is validating JDP’s prediction that people will focus less on alignment and more on misuse in their threat models of AI risk.
Fair point, valley9. I don’t think a little bit of agency throws you into an entirely different regime. It’s more that I think that the more powerful an agent you build, the more it is able to autonomously change the world to work with goals, the more you move into dangerous territory. But also, it’s going to tempt people. Somebody out there is going to be tempted to say, “go make me money, just don’t get caught doing anything illegal in a way that gets traced back to me.” That command given to a sufficiently powerful AI system could have a lot of dangerous results.
Indeed. This seems like more of a social problem than an alignment problem though: ensure that powerful AIs tend to be corporate AIs with corporate liability rather than open-source AIs, and get the AIs to law enforcement (or even law enforcement “red teams”—should we make that a thing?) before they get to criminals. I don’t think improving aimability helps guard against misuse.
I think needs to be stated more clearly: Alignment and Misuse are very different things, so much so that what policies and research work for one problem will often not work on another problem, and the worlds of misuse and misalignment are quite different.
Though note that the solutions for misuse focused worlds and structural risk focused worlds can work against each other.
Also, this is validating JDP’s prediction that people will focus less on alignment and more on misuse in their threat models of AI risk.