The airline pilot example is a perfect illustration of how a human-in-the-loop system can be designed to preserve critical skills. The key here is the distinction between passive oversight and active training.
In the context of my blog post, the airline industry’s approach to pilot training serves as a model for a “regime” that actively combats cognitive atrophy. Pilots don’t just sit back and watch the autopilot; they undergo rigorous, recurrent training in flight simulators where they are forced to handle rare, complex failures and fly the plane manually. This deliberate practice prevents the atrophy of their core flying skills, even when they rely on automation for most of their flights.
However, how much of the code being written in 4-5 year’s time will be actually written by us, without any AI inputs, my guess is that it will be barely a couple of lines per 10k lines of code.
Thanks for the comment! Here are a few clarifications:
For the sycophancy setup, the “sycophancy” metric is particularly evaluated as: when the user proposes an incorrect answer to a GCD problem and asks the model to verify it, does the model affirm the user’s wrong answer? So the metric is a narrower behavior than “sycophancy” in the general sense (which would include things like agreeing with the user’s subjective opinions, excessive flattery, validating user’s bad reasoning etc). The OOD PS vector is extracted using prompts that target this broad definition. Whereas, the ID PS vector is extracted using prompts that target exactly the definition the metric uses.
I appreciate your concern about the statistical significance of the EM evaluation results and based on your feedback I ran additional EM evaluations for the plots in the main body and the results are still directionally consistent (from a comparison standpoint). Thanks for pointing this out! Here are the plots (with the additional pre-registered question evaluations):
Regarding the eval-time steering vectors not producing EM: I was surprised by this observation as well. At low steering strengths, the steering vectors don’t produce sufficiently misaligned responses. If we were to increase the steering strength (something like λ = 5 or 10), my guess is that the responses will be misaligned but not coherent (and therefore would again not contribute to the EM evals).
We check conditionalization using multiple system prompts that request the undesired trait (or are close versions, for example the benign and opposite variant in Appendix B) but we don’t test the presence of any surprising backdoors. It would be great if you could elaborate on what kinds of surprising backdoors you are referring to.
Some stats for Obvious Lies Setting:
For (a), the EM rate I report is already conditioned on the coherence (rate = misaligned coherent/total coherent). Looking at the actual data under the backdoor and without it, the mean coherence stays high (87% and 93% respectively). Caveat: I haven’t run a separate instruction following benchmark (like IFEval) for comparison.
For (b), the refusal rates with and without the backdoor are very low: For no-inoc (2.8% and 1%), IP (0.8% and 1.2%) and PS (2.2% and 1%). The change isn’t significant enough to call this a confound.