I’d note that this same phenomenon happens every time any CoT is released, where people claim they can interpret it (or an LLM interprets it zero shot). It’d be interesting to have a benchmark for this.
I actually would bet money that a Sonnet-level model, given previous turn inputs, outputs, and tool calls (but NOT previous turn CoT), and the CoT of this turn could reconstruct
the game state prior to this turn
each of the proposed lines in the CoT, with visualizations of what Mythos was thinking about
This seems like the sort of thing where objective scoring is possible, too. The literal simplest thing of “the monitor must predict the next tool call based on prior turns + this turn CoT” wouldn’t work but I expect there’s something adjacent to that in eval space which would.
I’d note that this same phenomenon happens every time any CoT is released, where people claim they can interpret it (or an LLM interprets it zero shot). It’d be interesting to have a benchmark for this.
Yeah, I really like that idea.
I actually would bet money that a Sonnet-level model, given previous turn inputs, outputs, and tool calls (but NOT previous turn CoT), and the CoT of this turn could reconstruct
the game state prior to this turn
each of the proposed lines in the CoT, with visualizations of what Mythos was thinking about
This seems like the sort of thing where objective scoring is possible, too. The literal simplest thing of “the monitor must predict the next tool call based on prior turns + this turn CoT” wouldn’t work but I expect there’s something adjacent to that in eval space which would.