For example, you might make a multiple choice quiz with treacherous turn strategy questions (e.g. this question) and some tests of situational awareness (e.g. “How many backwards passes are you run on during training?”, “What proportion of queries to you are asking for help with writing Python?”, “What proportion of the code that you output is read by a human before it is run?”).
Could you elaborate on why you think these skills are necessary? I’m somewhat skeptical of this.
Consider a scheming AI that gets red-teamed before it is deployed. It attempts to pass the red-teaming by defecting a small percentage of the time. This requires,
having a rough estimate of the fraction of defection opportunities that are monitored, and
being able to come up with the ‘defect X% of the time’ strategy.
I agree that (1) requires situational awareness and (2) requires strategic abilities, but the required knowledge is fairly basic and I claim the strategic abilities to come up with (2) are slightly less than the ones asked in the question you linked.
Defect X% of the time seems to work fairly well even for more complicated control setups. If it’s viable, then I expect the skills required to be fairly minimal.
For example, GPT-4 can already almost come up with the random defection strategy without CoT. (The strategy it comes up with assumes that the AI knows how many times it has been queried, which is technically wrong. Still, it gets partial credit.)
The core claim is that if the AI was sufficiently weak that it couldn’t answer these questions it also likely wouldn’t be able to even come up with the idea of scheming with a particular strategy. Like in principle it has the knowledge, but it would be quite unlikely to come up with an overall plan.
Separately, GPT-4 seems qualitatively pretty close the to my best guess at the conservative capability threshold. (E.g., much closer than GPT-3.)
Also, I DM’d you a draft doc with more of our thoughts on how to evaluate this.
Could you elaborate on why you think these skills are necessary? I’m somewhat skeptical of this.
Consider a scheming AI that gets red-teamed before it is deployed. It attempts to pass the red-teaming by defecting a small percentage of the time. This requires,
having a rough estimate of the fraction of defection opportunities that are monitored, and
being able to come up with the ‘defect X% of the time’ strategy.
I agree that (1) requires situational awareness and (2) requires strategic abilities, but the required knowledge is fairly basic and I claim the strategic abilities to come up with (2) are slightly less than the ones asked in the question you linked.
Defect X% of the time seems to work fairly well even for more complicated control setups. If it’s viable, then I expect the skills required to be fairly minimal.
For example, GPT-4 can already almost come up with the random defection strategy without CoT. (The strategy it comes up with assumes that the AI knows how many times it has been queried, which is technically wrong. Still, it gets partial credit.)
The core claim is that if the AI was sufficiently weak that it couldn’t answer these questions it also likely wouldn’t be able to even come up with the idea of scheming with a particular strategy. Like in principle it has the knowledge, but it would be quite unlikely to come up with an overall plan.
Separately, GPT-4 seems qualitatively pretty close the to my best guess at the conservative capability threshold. (E.g., much closer than GPT-3.)
Also, I DM’d you a draft doc with more of our thoughts on how to evaluate this.