IMO, the main way things change if we assume that automated jailbreaking/persuasion attacks is the most likely strategy, compared to just attempting to communicate illegibly via steganography is that as in this experiment, paraphrasing CoTs almost entirely works to fix the monitor’s weaknesses without degrading any capabilities, because it’s way easier to separate dangerous reasoning from benign reasoning if the message itself is sent in plain text, and this remains true even if we cannot solve the issue of jailbreaking monitors or persuasion attacks.
More minorly, it hints that LLMs may have a similar issue to humans that Roger Dearnaley describes here:
It’s like they’re much less good at turning a description of an algorithm into an implementation if it has to be carried out in their initial and final layers than in their middle layers: it almost reminds me of the conscious and unconscious mind phenomenon in humans: except they seem to think they have conscious control when they don’t.
And depending on how much humans use neuralese, this capability deficit may persist even if CoT reasoning got replaced by huge neuralese models with memory banks, meaning neuralese is less bad for safety than feared.
It matters because if we don’t get steganography, it’s likely possible to fix monitors in ways that even superhuman AIs cannot avoid.
IMO, the main way things change if we assume that automated jailbreaking/persuasion attacks is the most likely strategy, compared to just attempting to communicate illegibly via steganography is that as in this experiment, [...] It matters because if we don’t get steganography, it’s likely possible to fix monitors in ways that even superhuman AIs cannot avoid.
But the whole problem here is that this particular setup (and indeed I’m arguing many of the “failed” attempts at creating a “what a narrow slice of humans specifically mean by ‘true’ steganography”) isn’t subject to optimization pressure such that it had to produce steganography. Once you introduce concepts like “fixing monitors”, you’re in an adversarial pressure regime where you’re absolutely going to get “thing that can get around your monitor”.
We demonstrate, for the first time, that unintended steganographic collusion in LLMs can arise due to mispecified reward incentives during training. Additionally, we find that standard mitigations—both passive oversight of model outputs and active mitigation through communication paraphrasing—are not fully effective at preventing this steganographic communication. Our findings imply that (i) emergence of steganographic collusion is a plausible concern that should be monitored and researched, and (ii) preventing emergence may require innovation in mitigation techniques.
For example, some of the semantic steganography in there ended up being particularly effective.
IMO, the main way things change if we assume that automated jailbreaking/persuasion attacks is the most likely strategy, compared to just attempting to communicate illegibly via steganography is that as in this experiment, paraphrasing CoTs almost entirely works to fix the monitor’s weaknesses without degrading any capabilities, because it’s way easier to separate dangerous reasoning from benign reasoning if the message itself is sent in plain text, and this remains true even if we cannot solve the issue of jailbreaking monitors or persuasion attacks.
More minorly, it hints that LLMs may have a similar issue to humans that Roger Dearnaley describes here:
And depending on how much humans use neuralese, this capability deficit may persist even if CoT reasoning got replaced by huge neuralese models with memory banks, meaning neuralese is less bad for safety than feared.
It matters because if we don’t get steganography, it’s likely possible to fix monitors in ways that even superhuman AIs cannot avoid.
But the whole problem here is that this particular setup (and indeed I’m arguing many of the “failed” attempts at creating a “what a narrow slice of humans specifically mean by ‘true’ steganography”) isn’t subject to optimization pressure such that it had to produce steganography. Once you introduce concepts like “fixing monitors”, you’re in an adversarial pressure regime where you’re absolutely going to get “thing that can get around your monitor”.
A good related paper is Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs:
For example, some of the semantic steganography in there ended up being particularly effective.