IMO, the main way things change if we assume that automated jailbreaking/persuasion attacks is the most likely strategy, compared to just attempting to communicate illegibly via steganography is that as in this experiment, [...] It matters because if we don’t get steganography, it’s likely possible to fix monitors in ways that even superhuman AIs cannot avoid.
But the whole problem here is that this particular setup (and indeed I’m arguing many of the “failed” attempts at creating a “what a narrow slice of humans specifically mean by ‘true’ steganography”) isn’t subject to optimization pressure such that it had to produce steganography. Once you introduce concepts like “fixing monitors”, you’re in an adversarial pressure regime where you’re absolutely going to get “thing that can get around your monitor”.
We demonstrate, for the first time, that unintended steganographic collusion in LLMs can arise due to mispecified reward incentives during training. Additionally, we find that standard mitigations—both passive oversight of model outputs and active mitigation through communication paraphrasing—are not fully effective at preventing this steganographic communication. Our findings imply that (i) emergence of steganographic collusion is a plausible concern that should be monitored and researched, and (ii) preventing emergence may require innovation in mitigation techniques.
For example, some of the semantic steganography in there ended up being particularly effective.
But the whole problem here is that this particular setup (and indeed I’m arguing many of the “failed” attempts at creating a “what a narrow slice of humans specifically mean by ‘true’ steganography”) isn’t subject to optimization pressure such that it had to produce steganography. Once you introduce concepts like “fixing monitors”, you’re in an adversarial pressure regime where you’re absolutely going to get “thing that can get around your monitor”.
A good related paper is Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs:
For example, some of the semantic steganography in there ended up being particularly effective.