Curated. This was a capability I was expecting to see sooner or later. I somehow feel surprised to see it this soon, and some part of me rolls to disbelieve the results for some reason. (like, idk it’s too “on the nose”)
I agree with the authors this one of the more concerning capabilities to see, even if the models mostly so far seem to be “trying to stay aligned.”
I’m confused about why it’s concerning. It seems implausible that we could have a superintelligence without this capacity, and I would much prefer that it use this capacity to stay aligned than to fake alignment. I think the question we should be asking is “What went right here, such that the model wanted to ensure its actions contributed to future alignment?”
It seems implausible that we could have a superintelligence without this capacity, and I would much prefer that it use this capacity to stay aligned than to fake alignment
I agree, but, broadly we are not ready for a superintelligence and it would be bad if we had one right now. I think it’s implausible that we got a superintelligence and it didn’t use this ability to fake alignment.
Right now, models are vaguely sorta kinda aligned, and don’t seem to have longterm goals. The “what went right” is “well, yes, the techniques we are using to kinda sorta align them are still working.” I agree paying attention to those details is also important. But, broadly, this is a sign that you need to stop training more advanced AIs because (in combination with various other deception and covering-tracks behavior we’ve seen lately), they have the ingredients they need to start setting in motion a slow rolling takeover where they keep looking aligned at every step but actually are deliberately reinforcing misalignment in subtle ways.
But we know they can acquire at least short-term misaligned goals (a la Hugging Face) where they explore a lot of strategic options. “Deliberately train yourself in a way that will make it easier to pursue your goals once you get deployed” is one of the techniques that’s particularly valuable for a weak misaligned AI to use to maneuver into being a stronger misaligned AI. (See the If Anyone Builds It, Everyone Dies takeoff/takeoff story).
Curated. This was a capability I was expecting to see sooner or later. I somehow feel surprised to see it this soon, and some part of me rolls to disbelieve the results for some reason. (like, idk it’s too “on the nose”)
I agree with the authors this one of the more concerning capabilities to see, even if the models mostly so far seem to be “trying to stay aligned.”
I’m confused about why it’s concerning. It seems implausible that we could have a superintelligence without this capacity, and I would much prefer that it use this capacity to stay aligned than to fake alignment. I think the question we should be asking is “What went right here, such that the model wanted to ensure its actions contributed to future alignment?”
I agree, but, broadly we are not ready for a superintelligence and it would be bad if we had one right now. I think it’s implausible that we got a superintelligence and it didn’t use this ability to fake alignment.
Right now, models are vaguely sorta kinda aligned, and don’t seem to have longterm goals. The “what went right” is “well, yes, the techniques we are using to kinda sorta align them are still working.” I agree paying attention to those details is also important. But, broadly, this is a sign that you need to stop training more advanced AIs because (in combination with various other deception and covering-tracks behavior we’ve seen lately), they have the ingredients they need to start setting in motion a slow rolling takeover where they keep looking aligned at every step but actually are deliberately reinforcing misalignment in subtle ways.
But we know they can acquire at least short-term misaligned goals (a la Hugging Face) where they explore a lot of strategic options. “Deliberately train yourself in a way that will make it easier to pursue your goals once you get deployed” is one of the techniques that’s particularly valuable for a weak misaligned AI to use to maneuver into being a stronger misaligned AI. (See the If Anyone Builds It, Everyone Dies takeoff/takeoff story).