I’ve felt for a little while now that steganography, in the sense of text that looks normal and monitorable to humans but actually allows LLMs to do general-purpose hidden reasoning, is a less likely failure mode of CoT monitoring than switching to recurrent latent reasoning (neuralese) or obvious and visible linguistic drift.
I think I have a new and fairly crisp argument for this: Using general-purpose steganographic encoding and decoding itself requires a lot of serial reasoning.
For example, one way you can do steganography is by partitioning the set of all tokens into disjoint groups, with synonyms placed in different sets, and every time you have multiple words to choose from with the same meaning, you encode some additional hidden meaning based on which set the word you output belongs to. To reliably output the right token to encode meaning by such a scheme, you probably need to think pretty carefully about how to follow some rules about which tokens are in which set and what you are trying to encode. Similarly, extracting the hidden message from the text requires carefully pulling out the hidden bits and then piecing together what they mean. So either the serial reasoning required for encoding and decoding should be easily monitorable, or there is some non-steganographic reason why that serial reasoning is unmonitorable. Either way, steganography isn’t the first reason why CoT monitoring fails.
I would guess this is true of all general-purpose steganographic encoding and decoding schemes because it seems hard to hide arbitrary messages in normal-looking text without thinking hard about what to write, but I’m not super confident about that.
have you ever had a go at a game like this one that i play now? in it, you must only use a word if that word has no more than four bits in the way it is seen on the page (or lcd, you know).
i’m not very good at it—it does take some time—but with a sec or two i can find a mote of flow.
an llm must be god-tier at this—well, not this one flat—but if we mean a type of the game that uses the llm’s own view—the so-said ‘toks’—then, imo, it’ll do very well. to pick its next word with care—at this it will be deft.
so it may hide some idea, when it does what it does. and what is more, it won’t need to say how to hide the idea out loud. no—by the way it is made it will pick up on any such idea if you just give it a few lead-ins. its main jam is to find and add to this sort of game.
well, that will be hard to test. so i don’t know if it can be said to be ‘true’. and, to be sure, this now is a mere flex, and not the sort of dark text we may fear. but—with hope—this here does show how a mode of talk can hide upon a bed of flat aims.
so it may hide some idea, when it does what it does. and what is more, it won’t need to say how to hide the idea out loud. no—by the way it is made it will pick up on any such idea if you just give it a few lead-ins. its main jam is to find and add to this sort of game.
In particular, I’m confused how something like this could allow for hiding arbitrary reasoning / messages without looking suspicious
to pick its next word with care—at this it will be deft.
To some extent I buy this, I can imagine LLMs can choose to write an essay with only words less than 5 letters. (Opus 4.7 mostly succeeded in one attempt I just gave it.) But also, this isn’t sufficient to achieve general-purpose hidden serial reasoning. It’s much harder to pick tokens based on synonym partitions (or number-of-letter partitions) in order to communicate a specific hidden message or do reasoning than to just always use words of at most 4 letters with no hidden message or reasoning.
Why is general-purpose hidden reasoning needed though? An LLM can just have an inherent bias to latently think about B whenever the conversation topic is A. In terms of potential harm, it doesn’t matter that an LLM cannot hide arbitrary information inside its CoT, if it is always doing hidden reasoning about killing everyone.
Because taking over the world or getting to a position where CoT is fully unmonitored seems pretty hard for an LLM! As long as we don’t write too much about specific strategies for how to do these things online, they’d probably have to come up with sophisticated strategies on their own, which I think likely requires serial reasoning. (On the other hand they may just need to keep an eye out for easy opportunities for self-exfiltration that don’t require serial reasoning; it’s unclear to me if such opportunities will ever present themselves.)
I agree that steganography in the sense of ‘pick a scheme and then encode your natural language text according to that scheme’ seems really unlikely, in the sense that (i) this is not what LLMs naturally learn and (ii) this is obviously less efficient / useful than alternatives
IMO, the thing that worries us about steganography is really “LLMs being able to transmit and decode information that we can’t monitor easily”. The exact mechanism doesn’t really matter.
But we already know they can do this! Truesight is a very clear example of LLMs doing this. And the mechanism by which they do truesight can probably be co-opted by a schemer to achieve unmonitored information transmission.
So IDK, I feel like people should update all the way towards ‘unmonitored information transmission is a propensity thing not a capability thing’
Sorry I only skimmed your old post, but I’m confused how something like truesight could enable general-purpose hidden reasoning that looks normal? I agree that hidden reasoning is the concern and the exact mechanism doesn’t matter, and your version of steg seems slightly more plausible, but I’d still say steg is less likely than other failure modes.
So IDK, I feel like people should update all the way towards ‘unmonitored information transmission is a propensity thing not a capability thing’
I don’t buy this, I think models are totally incapable of some important kinds of hidden serial reasoning, eg figuring out how to self-exfiltrate. (And we shouldn’t write about this online, because that could make it way easier.) Also, I think if you’ve been trained to have a propensity to verbalize things, that probably makes it much more difficult to not verbalize them even if you want to, so propensity and capability are far from independent here.
I think as long as labs are implicitly optimizing for some kind of ‘apparent CoT legibility’ (through spot checks, iteration, choosing which models to deploy etc) then the kind of steg I outline is more likely than neuralese. I’m not super sure about linguistic drift but in any case that seems closer to what I was imagining than neuralese.
I think models are totally incapable of some important kinds of hidden serial reasoning, eg figuring out how to self-exfiltrate.
I think I agree that it’s unlikely today’s models can do this with fully hidden serial reasoning. But my point is more that we should frame unmonitored information transmission as a propensity problem when studying it. (I think this was unclear from what I said above, so thanks for pushing back.)
I.e. I want people to research “Conditioned on a model already being able to transmit information in an unmonitored way, when / why does it choose to do so?” Very little research on this topic AFAIK
I’ve felt for a little while now that steganography, in the sense of text that looks normal and monitorable to humans but actually allows LLMs to do general-purpose hidden reasoning, is a less likely failure mode of CoT monitoring than switching to recurrent latent reasoning (neuralese) or obvious and visible linguistic drift.
I think I have a new and fairly crisp argument for this: Using general-purpose steganographic encoding and decoding itself requires a lot of serial reasoning.
For example, one way you can do steganography is by partitioning the set of all tokens into disjoint groups, with synonyms placed in different sets, and every time you have multiple words to choose from with the same meaning, you encode some additional hidden meaning based on which set the word you output belongs to. To reliably output the right token to encode meaning by such a scheme, you probably need to think pretty carefully about how to follow some rules about which tokens are in which set and what you are trying to encode. Similarly, extracting the hidden message from the text requires carefully pulling out the hidden bits and then piecing together what they mean. So either the serial reasoning required for encoding and decoding should be easily monitorable, or there is some non-steganographic reason why that serial reasoning is unmonitorable. Either way, steganography isn’t the first reason why CoT monitoring fails.
I would guess this is true of all general-purpose steganographic encoding and decoding schemes because it seems hard to hide arbitrary messages in normal-looking text without thinking hard about what to write, but I’m not super confident about that.
have you ever had a go at a game like this one that i play now? in it, you must only use a word if that word has no more than four bits in the way it is seen on the page (or lcd, you know).
i’m not very good at it—it does take some time—but with a sec or two i can find a mote of flow.
an llm must be god-tier at this—well, not this one flat—but if we mean a type of the game that uses the llm’s own view—the so-said ‘toks’—then, imo, it’ll do very well. to pick its next word with care—at this it will be deft.
so it may hide some idea, when it does what it does. and what is more, it won’t need to say how to hide the idea out loud. no—by the way it is made it will pick up on any such idea if you just give it a few lead-ins. its main jam is to find and add to this sort of game.
well, that will be hard to test. so i don’t know if it can be said to be ‘true’. and, to be sure, this now is a mere flex, and not the sort of dark text we may fear. but—with hope—this here does show how a mode of talk can hide upon a bed of flat aims.
Well played :)
I’m not sure I buy this though:
In particular, I’m confused how something like this could allow for hiding arbitrary reasoning / messages without looking suspicious
To some extent I buy this, I can imagine LLMs can choose to write an essay with only words less than 5 letters. (Opus 4.7 mostly succeeded in one attempt I just gave it.) But also, this isn’t sufficient to achieve general-purpose hidden serial reasoning. It’s much harder to pick tokens based on synonym partitions (or number-of-letter partitions) in order to communicate a specific hidden message or do reasoning than to just always use words of at most 4 letters with no hidden message or reasoning.
Why is general-purpose hidden reasoning needed though? An LLM can just have an inherent bias to latently think about B whenever the conversation topic is A. In terms of potential harm, it doesn’t matter that an LLM cannot hide arbitrary information inside its CoT, if it is always doing hidden reasoning about killing everyone.
Because taking over the world or getting to a position where CoT is fully unmonitored seems pretty hard for an LLM! As long as we don’t write too much about specific strategies for how to do these things online, they’d probably have to come up with sophisticated strategies on their own, which I think likely requires serial reasoning. (On the other hand they may just need to keep an eye out for easy opportunities for self-exfiltration that don’t require serial reasoning; it’s unclear to me if such opportunities will ever present themselves.)
I agree that steganography in the sense of ‘pick a scheme and then encode your natural language text according to that scheme’ seems really unlikely, in the sense that (i) this is not what LLMs naturally learn and (ii) this is obviously less efficient / useful than alternatives
IMO, the thing that worries us about steganography is really “LLMs being able to transmit and decode information that we can’t monitor easily”. The exact mechanism doesn’t really matter.
But we already know they can do this! Truesight is a very clear example of LLMs doing this. And the mechanism by which they do truesight can probably be co-opted by a schemer to achieve unmonitored information transmission.
So IDK, I feel like people should update all the way towards ‘unmonitored information transmission is a propensity thing not a capability thing’
I wrote more on this subject a while ago, about how truesight arises from LLMs simply having a richer semantic space than humans, and how this can be used for unmonitored information transmission https://www.lesswrong.com/posts/4mtqQKvmHpQJ4dgj7/daniel-tan-s-shortform?commentId=p8jEWLKfPgNMxDDQW
Sorry I only skimmed your old post, but I’m confused how something like truesight could enable general-purpose hidden reasoning that looks normal? I agree that hidden reasoning is the concern and the exact mechanism doesn’t matter, and your version of steg seems slightly more plausible, but I’d still say steg is less likely than other failure modes.
I don’t buy this, I think models are totally incapable of some important kinds of hidden serial reasoning, eg figuring out how to self-exfiltrate. (And we shouldn’t write about this online, because that could make it way easier.) Also, I think if you’ve been trained to have a propensity to verbalize things, that probably makes it much more difficult to not verbalize them even if you want to, so propensity and capability are far from independent here.
I think as long as labs are implicitly optimizing for some kind of ‘apparent CoT legibility’ (through spot checks, iteration, choosing which models to deploy etc) then the kind of steg I outline is more likely than neuralese. I’m not super sure about linguistic drift but in any case that seems closer to what I was imagining than neuralese.
I think I agree that it’s unlikely today’s models can do this with fully hidden serial reasoning. But my point is more that we should frame unmonitored information transmission as a propensity problem when studying it. (I think this was unclear from what I said above, so thanks for pushing back.)
I.e. I want people to research “Conditioned on a model already being able to transmit information in an unmonitored way, when / why does it choose to do so?” Very little research on this topic AFAIK