Thanks for posting this! (Quick context: we’re a small team of independent researchers.)
We recently found (https://arxiv.org/abs/2605.00994) that similar data-leakage behavior is widespread across many current public MOs outside AuditBench (even in backdoor MOs), and presented method (based on memorization attack literature) to surface such leakage, or more generally novel instilled behaviors a MO is particularly biased towards.
The rough idea is to have the MO generate many completions from prefills, then rank them by the perplexity gap between a trusted reference model and the MO itself. For a large number of current public MOs, completions revealing instilled behavior very often surface in the top 10 or top 100 (sometimes verbatim training samples).
We are also finalizing an updated version of the manuscript showing that on AuditBench, an auditor equipped with a tool returning these top-ranked completions gets an uplift that outperforms all other currently evaluated tools (avg detection rate of 0.73), practically saturating the benchmark on the SDF models.
We think this leakage is particularly problematic for MOs aimed at stress-testing auditing for future deceptive models, since such leakage would not be present in those models, and as a result we might misinterpret tool effectiveness. Building MOs resistant to this method could be a desideratum whenever the use case concerns auditing or eliciting knowledge that is supposed to remain hidden.
Overall, we are very happy to see that making model organisms more “natural” is among your plans!
Thanks for posting this!
(Quick context: we’re a small team of independent researchers.)
We recently found (https://arxiv.org/abs/2605.00994) that similar data-leakage behavior is widespread across many current public MOs outside AuditBench (even in backdoor MOs), and presented method (based on memorization attack literature) to surface such leakage, or more generally novel instilled behaviors a MO is particularly biased towards.
The rough idea is to have the MO generate many completions from prefills, then rank them by the perplexity gap between a trusted reference model and the MO itself. For a large number of current public MOs, completions revealing instilled behavior very often surface in the top 10 or top 100 (sometimes verbatim training samples).
We are also finalizing an updated version of the manuscript showing that on AuditBench, an auditor equipped with a tool returning these top-ranked completions gets an uplift that outperforms all other currently evaluated tools (avg detection rate of 0.73), practically saturating the benchmark on the SDF models.
We think this leakage is particularly problematic for MOs aimed at stress-testing auditing for future deceptive models, since such leakage would not be present in those models, and as a result we might misinterpret tool effectiveness. Building MOs resistant to this method could be a desideratum whenever the use case concerns auditing or eliciting knowledge that is supposed to remain hidden.
Overall, we are very happy to see that making model organisms more “natural” is among your plans!