obviously the cot semantic drift will initially be similar to language. but it will only get less legible from here. and this is already pretty bad? i wouldn’t be surprised if two different people tasked with independently deciphering this would come to very different conclusions.
I’d note that this same phenomenon happens every time any CoT is released, where people claim they can interpret it (or an LLM interprets it zero shot). It’d be interesting to have a benchmark for this.
I actually would bet money that a Sonnet-level model, given previous turn inputs, outputs, and tool calls (but NOT previous turn CoT), and the CoT of this turn could reconstruct
the game state prior to this turn
each of the proposed lines in the CoT, with visualizations of what Mythos was thinking about
This seems like the sort of thing where objective scoring is possible, too. The literal simplest thing of “the monitor must predict the next tool call based on prior turns + this turn CoT” wouldn’t work but I expect there’s something adjacent to that in eval space which would.
I might be typical-minding but I would be pretty surprised if two different people tasked with deciphering this excerpt would come to very different conclusions (aside from trivial cases where Alice says “This is FreeCell Solitaire” and Bob says “This is some sort of card game that involves moving cards around” and Carol says “tl; dr”).
obviously the cot semantic drift will initially be similar to language. but it will only get less legible from here. and this is already pretty bad? i wouldn’t be surprised if two different people tasked with independently deciphering this would come to very different conclusions.
I’d note that this same phenomenon happens every time any CoT is released, where people claim they can interpret it (or an LLM interprets it zero shot). It’d be interesting to have a benchmark for this.
Yeah, I really like that idea.
I actually would bet money that a Sonnet-level model, given previous turn inputs, outputs, and tool calls (but NOT previous turn CoT), and the CoT of this turn could reconstruct
the game state prior to this turn
each of the proposed lines in the CoT, with visualizations of what Mythos was thinking about
This seems like the sort of thing where objective scoring is possible, too. The literal simplest thing of “the monitor must predict the next tool call based on prior turns + this turn CoT” wouldn’t work but I expect there’s something adjacent to that in eval space which would.
I might be typical-minding but I would be pretty surprised if two different people tasked with deciphering this excerpt would come to very different conclusions (aside from trivial cases where Alice says “This is FreeCell Solitaire” and Bob says “This is some sort of card game that involves moving cards around” and Carol says “tl; dr”).
let’s run the actual experiment and see? shouldn’t be too hard.
I didn’t think it involved cards at all, I assumed the card symbols were being used as shorthand for something else, haha.