That seems like a much better example of illegible CoT, but I do wonder how much of that is just that I don’t know any Japanese and my knowledge of Shogi is “I think it’s kinda like chess but I have a vague memory that there’s something about a dragon”. And, based on reading this CoT, I infer that it’s played on a 9x9 board rather than an 8x8.
That said, Haiku knows Japanese and Shogi and (enough) web design—I wonder if it’d be enough to produce a verifiably faithful[1] annotated breakdown of this CoT. I’ll give it a shot.
BTW for reference there is indeed[2] a summarizer which sits between the user and the model’s raw CoT, translating the CoT into legible text similar to the post’s demonstration of Haiku translating the Solitaire CoT.
by “verifiably faithful” I mean “each snippet of shorthand has a bit of explanation mapped to it, there is no explanation that is not mapped to any bit of shorthand, the explanation as a whole makes sense, and each individual shorthand:explanation is a fairly obvious way to interpret that shorthand”.
The summarizer is a Haiku instance fed its previous summarized thinking and the current chunk and instructed to summarize that chunk in first person natural language. You can see the full summarizer prompt here (archive)
That seems like a much better example of illegible CoT, but I do wonder how much of that is just that I don’t know any Japanese and my knowledge of Shogi is “I think it’s kinda like chess but I have a vague memory that there’s something about a dragon”. And, based on reading this CoT, I infer that it’s played on a 9x9 board rather than an 8x8.
That said, Haiku knows Japanese and Shogi and (enough) web design—I wonder if it’d be enough to produce a verifiably faithful [1] annotated breakdown of this CoT. I’ll give it a shot.
BTW for reference there is indeed [2] a summarizer which sits between the user and the model’s raw CoT, translating the CoT into legible text similar to the post’s demonstration of Haiku translating the Solitaire CoT.
by “verifiably faithful” I mean “each snippet of shorthand has a bit of explanation mapped to it, there is no explanation that is not mapped to any bit of shorthand, the explanation as a whole makes sense, and each individual shorthand:explanation is a fairly obvious way to interpret that shorthand”.
The summarizer is a Haiku instance fed its previous summarized thinking and the current chunk and instructed to summarize that chunk in first person natural language. You can see the full summarizer prompt here (archive)