We might want to train models to robustly adhere to constitutions in context, but also be able to easily update the model later to fix imperfections in the constitution. This tension (robustness vs corrigibility) seems like a key tradeoff in alignment training.
It’d be great to measure the Pareto frontier here—e.g. which models are most robust and which are most corrigible? (Lots of people have this vague intuition that Opus 3 is really robust—can we quantify this?)
Maybe continual learning could push the frontier here—e.g. imagine swapping in new LoRA adapters / new MoE experts to add new principles to the constitution.
We might want to train models to robustly adhere to constitutions in context, but also be able to easily update the model later to fix imperfections in the constitution. This tension (robustness vs corrigibility) seems like a key tradeoff in alignment training.
It’d be great to measure the Pareto frontier here—e.g. which models are most robust and which are most corrigible? (Lots of people have this vague intuition that Opus 3 is really robust—can we quantify this?)
Maybe continual learning could push the frontier here—e.g. imagine swapping in new LoRA adapters / new MoE experts to add new principles to the constitution.