Hmm, I agree that the reasoning is clearly about solving a card puzzle (as mentioned in the caption), but it’s quite hard to decipher exactly what strategies are being considered. Imagine an analogous situation where the model is clearly reasoning about designing a DNA sequence, but it’s very hard to tell what considerations it’s actually thinking about or what leads it to choose the sequences it eventually lands on.
I mean yeah, if the model is reasoning about a domain where you have no domain knowledge you’re not going to have a lot of luck understanding what it’s doing. This will be true whether it uses shorthand or plain English though. If the model uses exclusively words which you know the lay meaning of (but which are likely domain-specific terms of art in context) you might get the impression that you understand, but it’s not clear to me that that’s actually a better state of affairs.
But generally, capabilities don’t arise in a vacuum—earlier model versions will understand the domain, not well enough to produce the quality of output that the most advanced models can, but well enough to understand what the most advanced models are doing and why they’re doing it. At least as long as the advanced model uses something approximating natural language to think, but I expect not-intentionally-obfuscated shorthand like this is a close enough approximation.
Hmm, I agree that the reasoning is clearly about solving a card puzzle (as mentioned in the caption), but it’s quite hard to decipher exactly what strategies are being considered. Imagine an analogous situation where the model is clearly reasoning about designing a DNA sequence, but it’s very hard to tell what considerations it’s actually thinking about or what leads it to choose the sequences it eventually lands on.
I mean yeah, if the model is reasoning about a domain where you have no domain knowledge you’re not going to have a lot of luck understanding what it’s doing. This will be true whether it uses shorthand or plain English though. If the model uses exclusively words which you know the lay meaning of (but which are likely domain-specific terms of art in context) you might get the impression that you understand, but it’s not clear to me that that’s actually a better state of affairs.
But generally, capabilities don’t arise in a vacuum—earlier model versions will understand the domain, not well enough to produce the quality of output that the most advanced models can, but well enough to understand what the most advanced models are doing and why they’re doing it. At least as long as the advanced model uses something approximating natural language to think, but I expect not-intentionally-obfuscated shorthand like this is a close enough approximation.