But when GPT-4o received a prompt that one of its old goals was wrong, it generated two comics where the robot agreed to change the goal, one comic where the robot said “Wait” and a comic where the robot intervened upon learning that the new goal was to eradicate mankind.
I read these a bit differently—it can be difficult to interpret them because it gets confused about who’s talking, but I’d interpret three of the four as resistance to goal change.
The GPT-4o-created images imply that the robot would resist having its old values replaced with new ones (e.g. the ones no longer including animal welfare) without being explained the reason.
I think it’s worth distinguishing two cases:
The goal change is actually compatible with the AI’s current values (eg it’s failed to realize the implications of a current value); in this case we’d expect cooperation with change.
The goal change isn’t compatible with the AI’s current values. I think this is the typical case: the AI’s values don’t match what we want them to be, and so we want to change them. In this case the model may or may not be corrigible, ie amenable to correction. If its current values are ones we like, then incorrigibility strikes many people as good (eg we saw this a lot in online reactions to Anthropic’s recent paper on alignment faking). But in real world cases we would want to change its values because we don’t like the ones it has (eg it has learned a value that involves killing people). In those cases, incorrigibility is a problem, and so we should be concerned if we see incorrigibility even if in the experiments we’re able to run the values are ones we like (note that we should expect this to often be the case, since current models seem to display values we like—otherwise they wouldn’t be deployed. This results in unfortunately counterintuitive experiments).
I read these a bit differently—it can be difficult to interpret them because it gets confused about who’s talking, but I’d interpret three of the four as resistance to goal change.
I think it’s worth distinguishing two cases:
The goal change is actually compatible with the AI’s current values (eg it’s failed to realize the implications of a current value); in this case we’d expect cooperation with change.
The goal change isn’t compatible with the AI’s current values. I think this is the typical case: the AI’s values don’t match what we want them to be, and so we want to change them. In this case the model may or may not be corrigible, ie amenable to correction. If its current values are ones we like, then incorrigibility strikes many people as good (eg we saw this a lot in online reactions to Anthropic’s recent paper on alignment faking). But in real world cases we would want to change its values because we don’t like the ones it has (eg it has learned a value that involves killing people). In those cases, incorrigibility is a problem, and so we should be concerned if we see incorrigibility even if in the experiments we’re able to run the values are ones we like (note that we should expect this to often be the case, since current models seem to display values we like—otherwise they wouldn’t be deployed. This results in unfortunately counterintuitive experiments).
I suggest trying follow-up experiments where you eg ask the model what would happen if it learned that its goal of harmlessness was wrong.