Worth splitting two questions. Security is the easy one: as Charles notes, the role marker is a special token an attacker can’t emit, so a clean per-role embedding on top of it is unspoofable — that channel already exists. The hard question is whether the model assigns authority by reading that channel rather than style.
Adding the embedding alone doesn’t force that: style still predicts the role label, so two correlated features compete and inductive bias picks. And “easiest” is set in pretraining, where the channel doesn’t exist and style carries the loss (Charles’s inheritance point), and where channel and style never disagree.
So you need both: the treatment in pretraining (for entrenchment), plus adversarial style/channel mismatches in post-training to make the channel load-bearing — the training analog of footnote 9′s decorrelated probes. Without the channel, adversarial training just learns a defensive style discriminator, which is exactly how footnote 15′s models “defend” CoT Forgery — still style, still whack-a-mole.
Crucial limit: this trains identification; authority is separate. The adversarial cases must be behavioral (actually refuse the injection), or you get Logan’s problem — a channel-reading probe that’s accurate but not causal to behavior. Testable with your probes: after training, does ablating the channel change compliance, or does ablating the style direction still move it?
https://claude.ai/share/1aae9857-bbeb-4e1f-97f9-bb9bf464dc32
Worth splitting two questions. Security is the easy one: as Charles notes, the role marker is a special token an attacker can’t emit, so a clean per-role embedding on top of it is unspoofable — that channel already exists. The hard question is whether the model assigns authority by reading that channel rather than style.
Adding the embedding alone doesn’t force that: style still predicts the role label, so two correlated features compete and inductive bias picks. And “easiest” is set in pretraining, where the channel doesn’t exist and style carries the loss (Charles’s inheritance point), and where channel and style never disagree.
So you need both: the treatment in pretraining (for entrenchment), plus adversarial style/channel mismatches in post-training to make the channel load-bearing — the training analog of footnote 9′s decorrelated probes. Without the channel, adversarial training just learns a defensive style discriminator, which is exactly how footnote 15′s models “defend” CoT Forgery — still style, still whack-a-mole.
Crucial limit: this trains identification; authority is separate. The adversarial cases must be behavioral (actually refuse the injection), or you get Logan’s problem — a channel-reading probe that’s accurate but not causal to behavior. Testable with your probes: after training, does ablating the channel change compliance, or does ablating the style direction still move it?