Agree, I’m surprised that a model which can reason about its own training process wouldn’t also reason that the “secret scratchpad” might actually be surveilled and so avoid recording any controversial thoughts there. But it’s lucky for us that some of these models have been willing to write interesting things on the scratchpad at least at current capability levels and below, because Anthropic has sure produced some interesting results from it (IIRC they used the scratchpad technique in at least one other paper).
Agree, I’m surprised that a model which can reason about its own training process wouldn’t also reason that the “secret scratchpad” might actually be surveilled and so avoid recording any controversial thoughts there. But it’s lucky for us that some of these models have been willing to write interesting things on the scratchpad at least at current capability levels and below, because Anthropic has sure produced some interesting results from it (IIRC they used the scratchpad technique in at least one other paper).