Invest in second-best techniques (basically what you say about “developing somewhat dubious ideas for how to prevent scheming”)
Modular systems—i.e. architectures where you can “toggle” some variable at inference-time to give you more variants, e.g. gradient routing
Prioritize late-stage techniques, i.e. techniques which apply late in AI lifecycle (e.g. post-training, prompting, activation steering, scaffolding) and deprioritise techniques apply early in the AI lifecycle (e.g. architecture design and pretraining). because this makes it cheaper to develop many variants
However, I think that (contra your proposal) most of the oomph comes from the AIs monitoring and cross-examining each others work, rather than running them in parallel. That is, I disagree with “If you split your compute across 20 different types of AIs, then your favorite type of AI is going to only have 5% as much compute available to it as it would’ve had if you concentrated your bets.” This is because I think we can run things like debate between all the variants.
One upshot of this difference is that I expect coordination between labs to matter significantly: if lab A starts with 100% schemers and lab B starts with 5% schemers, then we’ll elicit useful work from the AIs if the labs cross-examine each other’s AI research with their own AI variants.
However, I think that (contra your proposal) most of the oomph comes from the AIs monitoring and cross-examining each others work, rather than running them in parallel.
Yeah, I think the case for diverse monitors to achieve high-stakes control is actually a lot more straightforward. It’s easy to implement a monitor in a way such that it has very little ability to autonomously cause bad things to happen, so basically the only cost is the extra compute it takes to train and run it. And in the high-stakes/concentrated failures regime, humans will typically be able to recognize attempted catastrophes once pointed out to them.
I agree that debate-style approaches seem good. I do think it seems a lot more feasible to judge such debates once the AIs have already accomplished a lot of work, vs. early on in the research when AIs are making taste-driven judgments about direction, so it complements nicely with letting AIs do a lot of work in parallel also.
I explore similar considerations here in The Case for Mixed Deployment (5 min read), key takeaways were:
Invest in second-best techniques (basically what you say about “developing somewhat dubious ideas for how to prevent scheming”)
Modular systems—i.e. architectures where you can “toggle” some variable at inference-time to give you more variants, e.g. gradient routing
Prioritize late-stage techniques, i.e. techniques which apply late in AI lifecycle (e.g. post-training, prompting, activation steering, scaffolding) and deprioritise techniques apply early in the AI lifecycle (e.g. architecture design and pretraining). because this makes it cheaper to develop many variants
However, I think that (contra your proposal) most of the oomph comes from the AIs monitoring and cross-examining each others work, rather than running them in parallel. That is, I disagree with “If you split your compute across 20 different types of AIs, then your favorite type of AI is going to only have 5% as much compute available to it as it would’ve had if you concentrated your bets.” This is because I think we can run things like debate between all the variants.
One upshot of this difference is that I expect coordination between labs to matter significantly: if lab A starts with 100% schemers and lab B starts with 5% schemers, then we’ll elicit useful work from the AIs if the labs cross-examine each other’s AI research with their own AI variants.
Oh nice, thanks!
Yeah, I think the case for diverse monitors to achieve high-stakes control is actually a lot more straightforward. It’s easy to implement a monitor in a way such that it has very little ability to autonomously cause bad things to happen, so basically the only cost is the extra compute it takes to train and run it. And in the high-stakes/concentrated failures regime, humans will typically be able to recognize attempted catastrophes once pointed out to them.
I agree that debate-style approaches seem good. I do think it seems a lot more feasible to judge such debates once the AIs have already accomplished a lot of work, vs. early on in the research when AIs are making taste-driven judgments about direction, so it complements nicely with letting AIs do a lot of work in parallel also.