With that in mind, is there a clear criterion for dividing AI into ‘Teacher’ and ‘Student’ roles?
In general, the teacher will be the strongest model available to a lab that they don’t trust (else they’d just use that model). The student will be some weaker model that we’re much more confident we can trust, due to some combination of thinking it’s too dumb to be competently scheming against us, and having done heavy audits on it. In general these are referred to as U (for untrusted) and T (for trusted). We expect U to be broadly more capable than T, in the same way that Opus 4.8 is broadly more capable than Opus 3.
Furthermore, in Idea 4, you mentioned capping the Student AI’s intelligence at a level where alignment audits are still viable(i.e. auditable). It seems to me that we would still need some kind of metric to measure the LLM’s intelligence for this purpose. We can’t simply rely on benchmark scores, as we’ve seen far too many instances where they bear little relation to an AI’s actual ability to solve real-world problems.
Yeah this is a hard problem to solve. Basically we need to be able to quantify “how likely is it that this model could be subverting all our alignment evals”. Some ways we could do this are measuring capabilities required for doing that, like secret keeping, latent time horizon (i.e. ability to reason opaquely), situational/eval awareness, etc. If a model is bad at enough of these capabilities we might be reasonably justified in saying that it’s auditable.
In general, the teacher will be the strongest model available to a lab that they don’t trust (else they’d just use that model). The student will be some weaker model that we’re much more confident we can trust, due to some combination of thinking it’s too dumb to be competently scheming against us, and having done heavy audits on it. In general these are referred to as U (for untrusted) and T (for trusted). We expect U to be broadly more capable than T, in the same way that Opus 4.8 is broadly more capable than Opus 3.
Yeah this is a hard problem to solve. Basically we need to be able to quantify “how likely is it that this model could be subverting all our alignment evals”. Some ways we could do this are measuring capabilities required for doing that, like secret keeping, latent time horizon (i.e. ability to reason opaquely), situational/eval awareness, etc. If a model is bad at enough of these capabilities we might be reasonably justified in saying that it’s auditable.