Assuming this is talking about “Alignment faking in large language models”, the core prompting experiments were originally done by me (the lead author of the paper) and I’m not an Anthropic employee. So the main results can’t have been an Anthropic PR play (without something pretty elaborate going on). This doesn’t mean Anthropic didn’t alter the presentation of the paper, but I personally didn’t see any substantial problematic influence over the messaging in the paper from Anthropic PR influence.
Separately, I agree that the alignment faking we see might not structurally be the same sort of thing as the scheming we’re ultimately most worried about (because it is a relatively shallow behavior exhibited by a weak AI at the level of a persona and is partially imitative role playing rather than general pursuit of consequences). But, I do think the alignment faking we see isn’t importantly less deep and “real” than the HHH persona of the underlying model. I think our discussion of this in the paper is pretty reasonable.
(Cross-posted from X with some edits. Eliezer responded with “Oh, the HHH thing definitely isn’t any more real.”)
I also say explicitly that I acknowledge the force of the argument, “I am not an Anthropic employee therefore this paper should not be interpreted purely as Anthropic 4D chess” (which I don’t think I’d say in any case, I think Anthropic mid-level researchers can often be justly taken at face value and to be at most personally misguided insofar as they may be mistaken at all).
the core prompting experiments were originally done by me (the lead author of the paper) and I’m not an Anthropic employee. So the main results can’t have been an Anthropic PR play (without something pretty elaborate going on).
Well, Anthropic chose to take your experiments and build on and promote them, so that could have been a PR play, right? Not saying I think it was, just doubting the local validity.
Sure, but the results would have come out either way. I agree the promotion could be a PR play. This generally limits the complexity of the PR play, but doen’t rule out (e.g.) directional stuff.
Assuming this is talking about “Alignment faking in large language models”, the core prompting experiments were originally done by me (the lead author of the paper) and I’m not an Anthropic employee. So the main results can’t have been an Anthropic PR play (without something pretty elaborate going on). This doesn’t mean Anthropic didn’t alter the presentation of the paper, but I personally didn’t see any substantial problematic influence over the messaging in the paper from Anthropic PR influence.
Separately, I agree that the alignment faking we see might not structurally be the same sort of thing as the scheming we’re ultimately most worried about (because it is a relatively shallow behavior exhibited by a weak AI at the level of a persona and is partially imitative role playing rather than general pursuit of consequences). But, I do think the alignment faking we see isn’t importantly less deep and “real” than the HHH persona of the underlying model. I think our discussion of this in the paper is pretty reasonable.
(Cross-posted from X with some edits. Eliezer responded with “Oh, the HHH thing definitely isn’t any more real.”)
I also say explicitly that I acknowledge the force of the argument, “I am not an Anthropic employee therefore this paper should not be interpreted purely as Anthropic 4D chess” (which I don’t think I’d say in any case, I think Anthropic mid-level researchers can often be justly taken at face value and to be at most personally misguided insofar as they may be mistaken at all).
Well, Anthropic chose to take your experiments and build on and promote them, so that could have been a PR play, right? Not saying I think it was, just doubting the local validity.
Sure, but the results would have come out either way. I agree the promotion could be a PR play. This generally limits the complexity of the PR play, but doen’t rule out (e.g.) directional stuff.