Yes, R1 and Grok 4 do. QwQ does to a lesser extent. I would bet that Gemini does as well—AFAIK only Anthropic’s models don’t. I’m editing a paper I wrote on this right now, should be out in the next two weeks.
I’ve been suspecting that Anthropic is doing some reinforcement of legibility of CoT, because their CoTs seemed unusually normal and legible. Gemini too, back when it had visible CoT instead of summarized.
Also possible that Anthropic is actually giving edited CoTs rather than raw ones.
Anthropic says in their system card that Claude Sonnet 3.7 showed raw CoTs, and that Claude Opus 4 and Sonnet 4 show raw CoTs unless the CoT is especially long, in which case it is summarized. They also say this summarization happens about 5% of the time. According to the 4.5 system card, Claude Sonnet 4.5 reasoning text works the same way, but instead of giving a number, they say summarization happens in ‘a very small minority of cases’.
I agree that Anthropic and GDM may be reinforcing legibility in some way given how much more structured their CoTs look.
Does any other model have weird CoTs or just the OpenAI ones? If not, why not?
Yes, R1 and Grok 4 do. QwQ does to a lesser extent. I would bet that Gemini does as well—AFAIK only Anthropic’s models don’t. I’m editing a paper I wrote on this right now, should be out in the next two weeks.
I’ve been suspecting that Anthropic is doing some reinforcement of legibility of CoT, because their CoTs seemed unusually normal and legible. Gemini too, back when it had visible CoT instead of summarized.
Also possible that Anthropic is actually giving edited CoTs rather than raw ones.
Anthropic says in their system card that Claude Sonnet 3.7 showed raw CoTs, and that Claude Opus 4 and Sonnet 4 show raw CoTs unless the CoT is especially long, in which case it is summarized. They also say this summarization happens about 5% of the time. According to the 4.5 system card, Claude Sonnet 4.5 reasoning text works the same way, but instead of giving a number, they say summarization happens in ‘a very small minority of cases’.
I agree that Anthropic and GDM may be reinforcing legibility in some way given how much more structured their CoTs look.
Claude 4 system card: https://www-cdn.anthropic.com/6d8a8055020700718b0c49369f60816ba2a7c285.pdf#page=8
Claude Sonnet 4.5 system card: https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf#page=9
https://www.lesswrong.com/posts/FG54euEAesRkSZuJN/ryan_greenblatt-s-shortform?commentId=z7sxf8vGEu7E2Y5uW