Thanks for doing this monitoring and publishing the results!
I’m a bit concerned that you use GPT5.4 Thinking (max) for this monitoring task. How do you think about the obvious “but… we use the latest AI to monitor itself? If it really does start scheming, won’t it just decide not to snitch?” problem, and the more general problem of “the whole point of monitoring is that we don’t trust the AIs”
Also:
”That said, so far we have not seen evidence of motivations beyond the original task (for example, self-preservation or scheming).”
This sounds like you are saying “The models, they just want to complete the task” but if so, it’s clearly inconsistent with the evidence you provide later in the text. So perhaps what you mean is, “The models don’t seem to have any non-myopic goals, i.e. they don’t seem to care about what happens in the world long after the episode is over.”
I agree this is a concern, and we certainly need to watch out for collusion and measure this. I think on balance it currently makes sense to primarily be using our most powerful models as monitors (although in practice we monitor every trajectory multiple times with different monitors including older models and should continue to do so). Not least because the primary type of misalignment we see is an obsession with “completing the task”/seeming to do so.
Yeah the core point is that the goals seem myopic, perhaps it could have been phrased better.
Using multiple models sounds like a great approach. Older models should be cheap to add to the mix, and this way you get the best of both worlds (quality + difference), as well as an extra consistency check.
I’m not actually sure older models should be cheaper. GPT-5(.n) is amongst the cheapest models per-token that OAI have ever made. You could use N-minis, but those are distilled from N models which might have transferred misalignment. GPT-4 is definitely more expensive. Maybe GPT-3.5 is cheaper?
(And if you’re counting thinking tokens from 5.n models, you could just turn those off)
Have you tried to specifically validate your monitor using honeypots a la the existing work from Greenblatt et. al. in AI control, or do you think this would be redundant with the alignment evals GPT 5.4 has already been through?
Thanks for doing this monitoring and publishing the results!
I’m a bit concerned that you use GPT5.4 Thinking (max) for this monitoring task. How do you think about the obvious “but… we use the latest AI to monitor itself? If it really does start scheming, won’t it just decide not to snitch?” problem, and the more general problem of “the whole point of monitoring is that we don’t trust the AIs”
Also:
”That said, so far we have not seen evidence of motivations beyond the original task (for example, self-preservation or scheming).”
This sounds like you are saying “The models, they just want to complete the task” but if so, it’s clearly inconsistent with the evidence you provide later in the text. So perhaps what you mean is, “The models don’t seem to have any non-myopic goals, i.e. they don’t seem to care about what happens in the world long after the episode is over.”
I agree this is a concern, and we certainly need to watch out for collusion and measure this. I think on balance it currently makes sense to primarily be using our most powerful models as monitors (although in practice we monitor every trajectory multiple times with different monitors including older models and should continue to do so). Not least because the primary type of misalignment we see is an obsession with “completing the task”/seeming to do so.
Yeah the core point is that the goals seem myopic, perhaps it could have been phrased better.
Using multiple models sounds like a great approach. Older models should be cheap to add to the mix, and this way you get the best of both worlds (quality + difference), as well as an extra consistency check.
I’m not actually sure older models should be cheaper. GPT-5(.n) is amongst the cheapest models per-token that OAI have ever made. You could use N-minis, but those are distilled from N models which might have transferred misalignment. GPT-4 is definitely more expensive. Maybe GPT-3.5 is cheaper?
(And if you’re counting thinking tokens from 5.n models, you could just turn those off)
Have you tried to specifically validate your monitor using honeypots a la the existing work from Greenblatt et. al. in AI control, or do you think this would be redundant with the alignment evals GPT 5.4 has already been through?
A bit, but this is certainly something we should do more of.