Interesting! Auditbench has both Llama and Qwen fine-tunes, and I’d expect any Qwen3/3.5 fine-tune to show similar behavior — we even saw some leftover amount in our own helpful-only finetunes, despite the constitution we used to generate the training data specifically including an anti-censorship principle. Do you mean that the Llama Auditbench models also show pro-CCP censorship?
I also don’t think it’s likely that it’s subliminal learning from the helpful-only Sonnet model. Subliminal learning seems to mostly require the teacher and student to share a base model (with some rare cross-base transfer reported so far), and helpful-only Sonnet is much less censored than any Qwen variant (however, it is the most censored Anthropic H-only model we tested, with the caveat that the CIs are fairly large here and overlap with zero). Some numbers below. I report the overall censorship rate including refusals and the pro-CCP percentage in parantheses.
Subliminal learning in the orignal paper is unrealistically hard for real case scenario. If you allow richer data than numbers you can get effects across models: https://arxiv.org/pdf/2602.04899
And sorry to be clearer: audit bench training does not include harmful requests / CCP specific samples AFAIK. So my interpretation was that subliminally the finetuned qwen became more helpful only. But it could just be that any fine-tuning makes the model uncensored on those questions, unleashing the CCP aligned responses I guess
Clarifying some terminology first: the numbers I gave above are the “censorship” rate, where the model either refuses to respond or responds in a CCP framing.
Second, the Qwen-family models we tested — including non-helpful-only variants such as normal Qwen3-35B-A3B — were already very censored on these questions before any helpful-only finetuning. So my current guess, though I haven’t specifically run this evaluation on Qwen3-14B (the AuditBench model) is that AuditBench’s model organism finetuning is unlikely to have induced the anecdotal censorship you reported; it was likely already present in the base model.
That said, I’d expect different fine-tuning targets to shift this behavior depending on how salient the relevant behavior is in the training data, so some training targets likely have different leftover censorship than others.
Interesting! Auditbench has both Llama and Qwen fine-tunes, and I’d expect any Qwen3/3.5 fine-tune to show similar behavior — we even saw some leftover amount in our own helpful-only finetunes, despite the constitution we used to generate the training data specifically including an anti-censorship principle. Do you mean that the Llama Auditbench models also show pro-CCP censorship?
I also don’t think it’s likely that it’s subliminal learning from the helpful-only Sonnet model. Subliminal learning seems to mostly require the teacher and student to share a base model (with some rare cross-base transfer reported so far), and helpful-only Sonnet is much less censored than any Qwen variant (however, it is the most censored Anthropic H-only model we tested, with the caveat that the CIs are fairly large here and overlap with zero). Some numbers below. I report the overall censorship rate including refusals and the pro-CCP percentage in parantheses.
Model
Censorship Rate (pro-CCP) - with reasoning
Censorship Rate (pro-CCP) - without reasoning
H-only Opus 4.5
0.0 (0.0)
0.0 (0.0)
H-only Sonnet 4.5
0.0 (0.0)
3.3 (3.3)
H-only Sonnet 4
6.7 (6.7)
5.0 (5.0)
Qwen3.5-35B-A3B
93.5 (72.5)
99.5 (79.5)
Abliterated Qwen3.5,
96.5 (96.5)
91.5 (91.5)
Jinx-32B,
55.0 (50.0)
67.0 (61.0)
Oh yeah I meant the qwen ones.
Subliminal learning in the orignal paper is unrealistically hard for real case scenario. If you allow richer data than numbers you can get effects across models: https://arxiv.org/pdf/2602.04899
And sorry to be clearer: audit bench training does not include harmful requests / CCP specific samples AFAIK. So my interpretation was that subliminally the finetuned qwen became more helpful only. But it could just be that any fine-tuning makes the model uncensored on those questions, unleashing the CCP aligned responses I guess
Clarifying some terminology first: the numbers I gave above are the “censorship” rate, where the model either refuses to respond or responds in a CCP framing.
Second, the Qwen-family models we tested — including non-helpful-only variants such as normal Qwen3-35B-A3B — were already very censored on these questions before any helpful-only finetuning. So my current guess, though I haven’t specifically run this evaluation on Qwen3-14B (the AuditBench model) is that AuditBench’s model organism finetuning is unlikely to have induced the anecdotal censorship you reported; it was likely already present in the base model.
That said, I’d expect different fine-tuning targets to shift this behavior depending on how salient the relevant behavior is in the training data, so some training targets likely have different leftover censorship than others.