I agree that not acknowledging it at all would be counterproductive for the military. But I worry about obfuscation of the kind we saw in the Iran school bombing. They could acknowledge the initial error but not its mechanism. If the mistargeting is an outcome of misalignment (either actual power-seeking or just overeagerness), I have very little confidence that it’d be surfaced as such (edit: under current transparency standards), and not some hodge-podge of collateral damage and “a person did actually see it, that person made an error, our oversight was just bad—oops” and maybe “an AI system went a bit far here, but we’ve worked with the vendor to fix it and it is all good now, nothing to see here”
Omar Khursheed
Karma: 186
Clarifying some terminology first: the numbers I gave above are the “censorship” rate, where the model either refuses to respond or responds in a CCP framing.
Second, the Qwen-family models we tested — including non-helpful-only variants such as normal Qwen3-35B-A3B — were already very censored on these questions before any helpful-only finetuning. So my current guess, though I haven’t specifically run this evaluation on Qwen3-14B (the AuditBench model) is that AuditBench’s model organism finetuning is unlikely to have induced the anecdotal censorship you reported; it was likely already present in the base model.
That said, I’d expect different fine-tuning targets to shift this behavior depending on how salient the relevant behavior is in the training data, so some training targets likely have different leftover censorship than others.
Interesting! Auditbench has both Llama and Qwen fine-tunes, and I’d expect any Qwen3/3.5 fine-tune to show similar behavior — we even saw some leftover amount in our own helpful-only finetunes, despite the constitution we used to generate the training data specifically including an anti-censorship principle. Do you mean that the Llama Auditbench models also show pro-CCP censorship?
I also don’t think it’s likely that it’s subliminal learning from the helpful-only Sonnet model. Subliminal learning seems to mostly require the teacher and student to share a base model (with some rare cross-base transfer reported so far), and helpful-only Sonnet is much less censored than any Qwen variant (however, it is the most censored Anthropic H-only model we tested, with the caveat that the CIs are fairly large here and overlap with zero). Some numbers below. I report the overall censorship rate including refusals and the pro-CCP percentage in parantheses.
Model
Censorship Rate (pro-CCP) - with reasoning
Censorship Rate (pro-CCP) - without reasoning
H-only Opus 4.5
0.0 (0.0)
0.0 (0.0)
H-only Sonnet 4.5
0.0 (0.0)
3.3 (3.3)
H-only Sonnet 4
6.7 (6.7)
5.0 (5.0)
Qwen3.5-35B-A3B
93.5 (72.5)
99.5 (79.5)
Abliterated Qwen3.5,
96.5 (96.5)
91.5 (91.5)
Jinx-32B,
55.0 (50.0)
67.0 (61.0)
Agreed, edited to add more information from the linked post.